The Goldilocks Problem: Not too Much Data, Not too Little Data, the Challenge of Getting it Just Right with Dean Karlan

//April 11, 2013, CGCS presented a seminar with Dr. Dean Karlan.

Measurement’ and ‘impact’ are increasingly common buzzwords in global development.  Across all sectors, NGOs are scrambling to design monitoring and evaluation (M&E) systems that can demonstrate their programs’ impact on their target populations – but frequently without sufficient understanding of why or how this can be done. Oftentimes, data collection is ‘too big’ and yields information that cannot be accessed or used in any reasonable way. Data collection can also be ‘too small,’ leaving program managers with insufficient insight into how programs are performing. In the end, these efforts yield evaluations that provide un-reliable estimates of impact and monitoring systems that fail to support the day-to-day needs of program implementation.

The challenge facing NGOs is that while there is increasing pressure to ‘do M&E,’  there is no single answer to the question of how M&E should be done across the range of development sectors, approaches, organizations and contexts. The Goldilocks Problem proposes a framework for building ‘right-fit’ M&E systems, replacing the default impact imperative with two guiding principles: feasibility (‘Is it within the organization’s capacity to carry out this M&E activity fully and well?’) and actionability (‘Can the outputs of this M&E activity be used to inform decision making and action?’).  This approach does not replace the need for rigorous impact evaluation, but rather complements it by helping NGOs create M&E systems that can provide them with the information they need to promote effective development.

Dean Karlan is a Professor of Economics at Yale University. Karlan isPoverty Action, a non-profit organization that  creates and evaluates President  of Innovations for solutions to social and development problems, and  works to scale-up successful ideas through implementation and  dissemination to policymakers, practitioners, investors and donors.  Karlan is on the Board of Directors of the M.I.T. Jameel Poverty Action  Lab. As a social entrepreneur, He is Founder and President of  stickK.com, a website that uses lessons from behavioral economics to  help people reach personal goals, such as weight loss and smoking  cessation, through commitment contracts.

In 2011, Karlan co-authored More Than Good Intentions: How a New Economics is Helping to Solve Global Poverty. Karlan received a Presidential Early Career Award for Scientists and Engineers, and was named an Alfred P. Sloan Fellow. His research focuses on microeconomic issues of financial decision-making, specifically employing experimental methodologies to examine what works, what does  not, and why in interventions in microfinance, health, behavioral  economics and charitable giving. In microfinance, he has studied credit  impact, interest rate policy, savings product design, credit scoring policies, entrepreneurship training, and group versus individual liability. Karlan received a Ph.D. in Economics from M.I.T., an M.B.A. and an M.P.P. from the University of Chicago, and a B.A. in International Affairs from the University of Virginia. He can be followed on twitter @deankarlan, and blogs regularly on Freakonomics.

Leave a Reply