Why impact measurement is all about power and money
Impact measurement is a tool that can be used to redistribute power, but it can also be used to concentrate power among those with the most money. Tris Lumley, director of development at New Philanthropy Capital, who is currently chairing the working group on impact measurement in the Social Impact Investment Taskforce, takes a look at how power and money flows in and around organisations doing social good.
I am an evangelist about impact measurement for charities and social enterprises. Not because I’m a geek, though that’s certainly true. But because impact measurement is a tool for influencing the flow of money and power.
My problem is that all the effort around impact measurement is driven by a need to meet funders’ requirements, not to learn about and improve the work. It’s driven by accountability to funders, not accountability to the people being served. That would be ok if funders’ requirements were very closely aligned to what’s best for beneficiaries, but in the vast majority of cases they aren’t. I’ve lost count of the number of times that front-line staff have told me that they hate administering this or that survey, but they have to “for the funders”.
Individual donors may be interested in impact, but they’re more likely to ask about what percentage of their gift will get to the front line, rather than being spent on ‘overheads’. Trusts and foundations may ask about outcomes, but they’re less likely to back that up with support to the organisation to develop an evaluation framework that helps them develop learning and continuous improvement. Council grants or procurement officers may be interested in results, but most of the time they’ll actually focus on costs, outputs, and other data that’s not of great use to the organisation. Social investors may be interested in outcomes, but much of the time they’ll focus on where the revenue is coming from. The social impact may just be a hurdle to overcome early in the due diligence, after which the focus will be on the finances.
If we zoom out to think about what’s going on, what we’ve really got is relationships, power and money. Funders have money, so they have power over social purpose organisations that may apply to them for that money. Once the social purpose organisation has money, it has power over beneficiaries, who rely on their programmes and activities.
If impact measurement is done well, it redistributes power in these relationships. If it captures accurate accounts of the real changes in people’s lives that occur (partially) as a result of an organisation’s work, it allows an accurate narrative to be told to those with the money, to help guide their decisions. It allows an organisation to learn about what’s effective in its work and what could be improved. Like the first impact report of the Fifteen Foundation, which was quite clear about the programme’s failure to properly accommodate the difficult backgrounds and contexts of some participants, resulting in them dropping out. The report still stands as a rare example of learning from failure and sharing the lessons .
Good impact measurement is a feedback loop based on what happens in real people’s lives. It results in services and products being improved, and money flowing to services that are effective and highly valued. And it ultimately results in people having more of a stake in the products and services offered to them—their voice being heard and acted on. For example, when Royal Voluntary Service carried out a warts and all impact study in 2008, it found that its services were sometimes perceived as inflexible, and its community centres among the least effective of its service areas. Since then, it has changed in response, becoming more customer-focused and flexible, and trying to work out what people want and need before offering them a service.
Bad impact measurement tells stories to access money, regardless of what happens in real people’s lives. Any organisation can collect the positive case studies that make its programmes seem incredibly powerful, but fail to represent the true picture which may be much more equivocal. For example, of the 13 accommodation programmes whose results were published by the Justice Data Lab in June 2014, seven had no statistically significant impact, three had a positive impact, and three a negative impact.
My point? Impact measurement is just a tool, and as with any other tool, it can be used for good or ill. It can be used to distribute power—to empower—but it can also be used to concentrate power. Philanthropy can be seen as act of exercising power—the person with the money gets to choose what happens to those without it.
At its best, impact measurement is a check on this power. It gives us a picture of the world that helps us make better decisions, and ultimately give power to those that don’t have it.
This article was produced in partnership with the SROI Network following the annual SROI Network event, Social Value Matters.
Photo credit: Philip Henzler, Unsplash.