[caption id="attachment_3374" align="alignright" width="327"] Beneficiarios de proyecto visitado en Ecuador[/caption]

However, despite its importance, this is still not a common practice in the sector.  Instead, many impact reports focus narrowly on counting the total of units distributed or the number of beneficiaries supported. This practice is insufficient and defeats the purpose of what a true “impact measurement” should be: a source of valuable insights about the benefits being transferred, their magnitude and their potential for increase.

Reports including data on the number of people reached and featuring heart-warming stories about a few cases where the beneficiaries’ lives were improved as a result of the organization’s intervention (a common practice in this sector) can certainly serve for marketing purposes. However, they can also be misleading in overstating the organization’s true impact, since there is no way to distinguish whether the experiences of the people highlighted in these stories are shared by the majority of the targeted group—or if they are just outliers whose outcomes surpassed the rest, either by pure luck or other unrelated factors.

But how can we find out if the rest of the beneficiaries are really better off because of us? Individual results may vary and every person’s experience is unique and inimitable. But is it at least possible to determine whether results were positive for a significant portion of them? Understanding any outcomes that are attributable to our intervention is paramount, not merely for supervision purposes or as a marketing/funding tool, but also for conducting useful analyses that can then be used to improve the effectiveness of our impact and our operating efficiency.

To accomplish this, we need to truly make an effort to understand the end beneficiaries and the different ways they interact with the product/service provided, instead of merely trying to count them (which is only a modest first step).

For social enterprises, monitoring and evaluation methods can be classified into two opposing extremes. On one hand, we have management information systems with standardized metrics that serve as the source for the kinds of reports mentioned above. These instruments are typically limited in nature and rely almost exclusively on self-reported data from clients. Because of it, they are in many cases barely suitable to accurately count the number of beneficiaries reached on the ground, let alone make any claims about the benefits that were (supposedly) transferred to them.

On the other hand, we have impact evaluations and their trendier siblings RCTs (Randomized Control Trials), which are generally considered as the most credible means of accurately identifying the impact of any social intervention. During the last decade, they have enjoyed broad acceptance as tools for ascertaining what works and what doesn’t in the development field and for disproving myths and common preconceived notions. Their main drawback, however, is that they can be quite expensive and take several years to implement (if everything goes according to plan).

The ultimate goal should be for impact measurement practices to serve as a middle ground between these two poles: a sort of sweet spot where we could rely on readily available information compiled on a rolling basis by portfolio companies, but still be able to say something insightful about the benefits we are transferring to people. Nevertheless, it seems that currently very few efforts are bridging this gap. To understand why, we need to take a closer look at the rationale behind the data that is being requested from investee companies, which focuses mainly on tracking the use of funds and determining how many lives were touched. Yet from an analytic point of view, responding this question offers very few insights.

In an effort to find the middle ground, I traveled recently to Ecuador with the purpose of expanding the level of granularity in the performance reports we were getting from a financial institution (FI) in our portfolio that works with vulnerable populations in rural areas. After meeting with their accounting department and reviewing their internal databases, I was able to make an initial assessment on their reporting capabilities. I discovered a vast array of useful client-level data which was unknown to us until that point, and could serve as complimentary to better understand the effect of the FI’s operations.

The FI fostered a very close relationship with its clients and therefore had more information on them and their economic activities than usual, which turned out very useful in enriching our analysis at the outcome level. Additionally, because of this close connection, the FI could also easily ask clients for feedback after they repaid their loans, which could allow us to start having two data points for each borrower on relevant indicators such as income level and saving amounts. For their infrastructure projects, we were also able to obtain social data on the number of people who benefitted by each project, and in some cases, even data that hinted towards increases in education skills and job placement rates. We also were able to find some efficiency disparities between projects, when we compared the different costs among projects that were for very similar uses and scales.

We might never use this outcome-level data to run a full-fledged impact evaluation, and will therefore never be able to really ascertain the causal link between the FI’s work and these observed results. Nevertheless, the fact that we were able to get this data in such a cost effective way will nonetheless help make our assessment come full circle and improve our ability to evaluate this business’s social impact.

In a sector dominated by investment bankers and corporate finance specialists, it is important that we strive to promote a better understanding of the nuances and inner workings of creating social impact and reject views that consider it merely a prêt-à-porter additive to usual business, even in those instances where the targeting was done correctly. After all, the harsh reality is that social impact is not created merely by reaching the poor, and counting the number of poor people reached is not enough to prove it.

We need to do a better job at comprehending the complex trade-offs between social and financial returns. We need to make sure that when our products/services reach these vulnerable populations, they do more good than harm in the long run. We need to be certain the benefits are sustainable. Finally, we need better proof of what works and what doesn’t and we need to learn from our mistakes. This is still a nascent industry, and we have a long way to go.

In order to encourage the discussion around this and other topics related to impact investing and the business and development potential of base of the pyramid markets, the IDB is holding the BASE III Forum in Mexico City on June 29-July 1 at the Centro Banamex.