Microfinance Newsletter Image of women working UNCDF logo 2005: Year of Microcredit
colorful bar

UNITED NATIONS CAPITAL DEVELOPMENT FUND    Microfinance

Issue 15 / August 2005

     

Past Issues

Understanding Pitfalls in Measuring the Impact of Microfinance:

An Interview with Jonathan Morduch, Associate Professor of Public Policy and Economics, Robert F. Wagner School of Public Service, New York University, and author of the book, The Economics of Microfinance*

  1. In the preface of The Economics of Microfinance, you state that "microfinance successes force economists to rethink assumptions about how poor households save and build assets, and how institutions can overcome market failures." What have you observed in your studies and on the ground that inspired you and Beatriz Armendáriz de Aghion to reframe old debates within microfinance?

    We both started writing the book with a mixture of practical and academic experiences. Beatriz is a founder of the Grameen Trust in Chiapas, Mexico, and she has made important contributions to the theory of banking and microfinance. Over fifteen years, I have worked on financial issues in both Asia and Latin America, and I have taken part in (sometimes fractious) policy debates among donors. One aim of the book is to put the rich insights of practitioners and donors into conversation with the broader insights of academic economists.

    Our experience on the ground, for example, leads us to argue that our colleagues in academia have paid too much attention to borrowing constraints and too little to difficulties that households have in saving. The challenges faced by households in managing small and irregular income flows have also been under-appreciated. Perhaps most controversially, we show how group lending in practice works very differently (and less well) than group lending as described by theorists.

    In the other direction, we found that many practitioners and donors are seeking structured ways to think through incentive problems and to unpackage financial mechanisms that had been developed elsewhere and taken off the shelf. Academic frameworks have a lot to add here. We also try to cut through the debates around impact evaluations and the role of subsidies.

    Much on microfinance has been written by donors and practitioners who advocate particular approaches. They have brought passion and rich descriptions of their own experiences, but, with a few important exceptions, they have seldom tried to provide balanced views. One of our aims is to root our arguments in the best available evidence (even when it's contradictory or issues are unresolved), and we make clear when we simply lack good numbers. In the end, pointing out what we genuinely don't know (the list is sometimes surprising) may be as important as pointing out what we do.

  2. You stress that microfinance is a poverty-reduction tool capable of standing on its own, free from the burdens of large state subsidies. You also note, however, that subsidies can be used to create efficiency benchmarks without undermining the integrity of the institution. What are the problems associated with government subsidies and how should microfinance institutions approach the use of subsidies with respect to the goal of becoming independently profitable banking institutions?

    Subsidies have always been a part of the landscape of microfinance, and institutions use them in different ways. Subsidies are used to help defray start-up costs, to upgrade information systems, to reach costly-to-serve target groups, or to add health inputs or education along with financial services. Even profit-making banking institutions take advantage of subsidies for particular purposes, like experimenting with new products.

    The interesting question is not whether subsidies are good or bad, but whether they can be used in smart ways to generate new opportunities. Experience shows that well-designed subsidies can foster innovation and expand the scale of microfinance. BRAC's Targeting the Ultrapoor program in Bangladesh, for example, uses subsidies for an intensive and time-limited period to help develop the skills of potential customers. The target group has motivation but lacks the human capital and economic stability to borrow effectively. The subsidies allow BRAC to expand its scale and to eventually work with poorer-than-average customers in a sustainable way.

    At the same time, badly-designed subsidies-whether from governments, international donors, or individual social investors-can undermine incentives, create dependence, and limit innovation. Avoiding these problems requires a commitment to transparency and to achieving efficiency (even when profitability is not in hand). Donors can help by tying funding to the achievement of specific efficiency targets.

  3. In Chapter Eight of The Economics of Microfinance, you address the difficulty of measuring the impact of microfinance programmes as a result of various biases. What are some examples of the hindrance imposed by these biases on measuring the real impact of a product or service?

    Measuring impacts need not be complicated or expensive. But it does require an understanding of potential pitfalls. The starting point is recognition that microfinance customers are seldom typical of their communities. From a banking standpoint, choosing especially promising customers is something to celebrate, but it raises hurdles when it comes to measuring impacts. Recent studies in Thailand, Peru, and Bangladesh, for example, each show how microfinance customers tend to already start with more wealth or a greater sense of "empowerment" and personal autonomy than their neighbors. Thus, even without access to microfinance, these households would be doing better than their neighbors. Impact studies that fail to account for these kinds of initial advantages exaggerate the role played by microfinance in transforming lives.

    The good news is that it's straightforward to address the potential biases. The bad news is that, in practice, there's still a long way to go in improving impact studies. One recent survey of 17 microfinance institutions around the world, for example, asked about financial and social measurement. Nearly all institutions had completed an impact survey or were going to do so soon. But only one had included a control group in their study. Without a control group, it is difficult to obtain robust and realistic impact estimates.

  4. That most impact evaluations in microfinance do not measure unobservable characteristics of clients such as entrepreneurial spirit and business connections often leads to a skewed perception of the effectiveness of a microfinance programme. How exactly are evaluations affected by these unobservable traits and what can microfinance institutions do to correct these problems?

    The most direct way to address the potential biases is to collect data on control groups-i.e., households that don't now have access to microfinance. The big question then is how do households with access to microfinance differ over time from those who don't?

    So, how should control groups be chosen? Using "new borrowers" and comparing them to "older borrowers" is a simple idea, but raises potential problems. After all, why did some borrowers sign up first before the others? And are the older borrowers a special group by virtue of staying in the program and not dropping out?

    Researchers at Freedom from Hunger implemented a better solution in their studies of affiliates in Ghana and Bolivia. The institutions in both places were set to expand substantially over a period of several years. The institutions agreed to identify the potential new locations ahead of time and to choose where to start by essentially flipping a coin. Since the institutions were indifferent to the order of expansion, they lost nothing through the process. The researchers, though, gained a big advantage since the order of expansion was randomized. The researchers surveyed all the villages prior to the expansion and then again afterward. The researchers could then ask whether households in villages with early access to microfinance did better than those with delayed access. The findings showed that places with early access had higher levels of economic security and asset accumulation, as well as improvements in the status of women and children (http://www.ffhtechnical.org/cwep/impact.html).

  5. In your book, you maintain that finding instrumental variables that accurately measure credit programmes has been challenging. What are instrumental variables and how can the microfinance sector successfully incorporate the use of these variables in impact assessments to achieve more accurate results?

    The job of researchers who worry about proving the causal impact of microfinance is to find ways to cut through the biases described above. The traditional starting point is to find a variable that helps to explain why some people get access to finance but others don't. One example is the eligibility rule for access to microfinance in Bangladesh-many institutions restrict access to households that own no more than a half-acre of land. Those households with just above the land cut-off should (in principle) be denied access. To be useful as an "instrumental variable," the variable should also have no direct impact on outcomes of interest (like income, assets, empowerment, etc.). When both conditions are met, researchers can use specially-designed (and commonly-used) statistical methods that create clean estimates of causal impacts.

    The approach is used in much economic research, but it turns out that the second requirement is hard to meet in the microfinance context. A more promising idea is to capture the flavor of the approach by designing expansion strategies with evaluations in mind--the idea adopted by Freedom from Hunger and described above.

  6. The issue of drop-outs has thwarted precise impact assessment in the microfinance sector. How do drop-outs affect evaluations and furthermore, what potential solutions can microfinance institutions use to overcome this obstacle?

    Microfinance is not for everyone, and customers drop out over time, even from the most successful institutions. Drop-out rates have been reported as high as 25-60 percent per year in East Africa, and below 5 percent per year in Bangladesh. One survey in Bolivia found that half of the respondents who ever borrowed from BancoSol were no longer active customers by 1996.

    Sometimes dropping out signals success-the customer has adequate assets now and no longer needs the help of a bank. Other times, drop-outs arise after defaults or after a realization that borrowing isn't bringing hoped-for returns. The "survivors" will then be a skewed sample. As noted above, comparing "older" active borrowers to "new" borrowers will then introduce potential biases.

    The obvious solution is to track down drop-outs and include them in the survey along with other "older" borrowers. A cheaper, next-best solution is to create a statistical profile of drop-outs and use that to adjust estimates. The methods are not complicated, but life is made easier by taking the potential problems into account before studies get underway, rather than trying to fix problems late in the game.

*To purchase The Economics of Microfinance, please visit the International Year of Microcredit's Made by Microentrepreneurs online store: http://www.shopmicro.org/

Praise for The Economics of Microfinance:

"The single best book on the economics of banking and finance, period, and certainly the most encompassing book I have read on microfinance. My copy is covered in notes and dog-eared from use."

- Tom Easton, New York Bureau Chief, The Economist