How do we stop impact metrics from becoming counterproductive?
Spend too little time tracking the efficacy of development programmes and you risk wasting resources, not to mention losing the confidence of your backers. But spend too much time and you could still end up wasting resources, thus doing less of the good you initially set out to do. Christian Jahn and Susann Tischendorf from the Inclusive Business Action Network on how to approach the 'Goldilocks dilemma' of tracking impact.
While this signals positive public support for the work of aid and development agencies around the world – and for the prominent role of EU countries in funding existing development work – there is a challenge in translating this support into action.
Most organisations, both public and private, have to justify their programmes to taxpayers or donors, leading to extensive attempts to measure the impact of development work.
Yet collecting and evaluating data to do this can take a great deal of time and effort – sometimes at a detriment to the core programme goals.
Thus, many organisations funding global development face a Goldilocks dilemma. If they spend too little time tracking the efficacy of their work, they could waste resources and lose the confidence of their backers. But if they spend too much time on it, they could still end up wasting resources and doing less of the good they initially set out to do.
To solve this problem, we need ways of assessing impact that are rigorous enough to justify development activities to public and private funders, while also being simple enough not to require a disproportionate investment of resources.
Collecting and evaluating data can take a great deal of effort – sometimes at a detriment to the core programme goals
For Rabayl Mirza from the UN Development Programme’s Business Call to Action alliance, the key is to focus on reaching well-established development goals like the UN Sustainable Development Goals (SDGs) – but not within a silo. The alliance advises partners to assess project impacts against the 17 SDGs, through organisations such as Global Reporting Initiative and SDG Impact, so that results can be aggregated and compared across initiatives and companies.
The alliance’s own approach is based on the five dimensions of impact developed by the Impact Management Project: what the project is; who is involved; how much is being invested; the contribution made by relevant stakeholders; and the risks that may be present in the project. By studying the operations of the company, it is possible to effectively illustrate how they contribute over time to outcomes related to the SDGs.
This development-focused approach can be accused of having shortcomings, however. One problem is that well-established goals may not be a good fit for a project, or the goals themselves may not adequately cover the aspirations of a project. For example, Bill Drayton, founder of social entrepreneurship organisation Ashoka, claims that tracking progress in this way can be largely illusory and that a goal like solving unemployment in Africa is never going to be achieved just by giving young people skills.
Rather than focusing on codifying goals, some argue the impetus should be on building the capacity of individuals and organisations to instigate change when they encounter a social problem.
Indeed, Drayton believes that individual capacity should be the priority for assessing impact. Every organisation and individual should look at themselves and others as “changemakers” and feel that they have the support to improve the world as they see it.
But some say this capacity-focused approach does not provide an accurate measure of success or sufficient guidance for making decisions in the heat of the moment.
This could be tackled by concentrating specifically on the market being served.
For instance, the metrics from USAID’s Securing Water For Food programme may appear to tell the whole impact story – with almost 20bn litres of water saved and 6.3m end users benefitting from results such as increased crop yields. Yet, according to Dr. Ku McMahan, USAID’s team lead for the project, success is not always so tangible and numbers paint only part of the picture.
Dr. McMahan asks, “Were more women empowered in their communities because they are now taking an active part and not simply keeping the house but helping grow crops? Has an innovation increased a family’s income so a child could go to school where before there simply was not enough money?”
Project impact can be assessed through the need it fulfils and how badly it is wanted
By looking at a problem from the perspective of a programme’s beneficiaries in this way – and questioning the impact that a development programme has had on the day-to-day lives of individuals and families – project impact can be assessed through the need it fulfils and how badly it is wanted.
“We need to recognise metrics are only as good as the integrity of the data going into their calculation and the degree to which we understand their purpose,” notes Jed Emerson, who developed the Blended Value approach – designed around evaluating organisations based on a mix of financial, social and environmental value.
Working towards a shared framework of impact measurement is key, Emerson argues, but goals should not be confused with actual progress on the ground.
Regardless of whether metrics are focused on development, capacity or markets, it is vital to remember the reason behind tracking them in the first place. The purpose is to work together towards achieving real, lasting impact.
Christian Jahn is head of the Inclusive Business Action Network (iBAN) secretariat; Susann Tischendorf leads communication and digital innovation at iBAN.
Thanks for reading Pioneers Post. As somebody working in the impact economy, you'll know that producing quality work doesn't come free. We rely on paid subscriptions and partnerships to sustain our purpose-led journalism – so if you think it's worth having an independent, specialist media platform to share your news, insight and debate across the globe, please consider subscribing. You'll also be buying social: Pioneers Post is a social enterprise itself, reinvesting all profits to help you do good business, better.