Measure me this

Over the past few years, the buzz on the street in the development space (read: international development) has been all about impact. What is it? How does it affect our bottom line? Is it related to success – and if so, how? Who’s involved? How to improve it? But most importantly:

Quite a few people have been thinking about this for a while now, and first among them is Sasha Dichter from Acumen Fund. His thoughts on the matter merely reflect a broader movement within philanthropy that is focusing on what it means to create a sustainable development space that is responsive to changes on the ground. Of course, before measurement comes definition, and what a bugbear that has proved to be! Sara Olsen at the SVT Group has spent years trying to answer this question, and while she has been generally successful in some ways, it seems that a cookie cutter solution will be hard to come by.

I also had a great conversation a few months back with Brad Presner, metrics manager at Acumen, where we discussed the ins and outs of impact measurement, and how I had noticed that very few people in philanthropy are actually asking the end users, in this case the recipients of funding and development aid, to measure development impact as it appears to them. From an experience designer’s perspective this seems pretty self evident: go to the source (i.e. the end user) when designing a solution, because that’s where you’ll get the most valuable feedback. If you take it one step further by implementing a continual feedback loop, you end up with a more agile, relevant end product, no matter what the sector.

The missing link

This lack of a feedback loop isn’t a new problem to the international development space. While at Origo and doing work for USAID, Kellogg Foundation and others, one of the most common requests we had was “Help us measure impact!” We tried everything: from creating a mapping framework for USAID missions, to graphically recording Kellogg events in an attempt to draw new insights from the visual medium, to being involved with the GIIN network and its IRIS measurement tool back when it was an early prototype. All seemed very promising, and indeed, some (like IRIS) are gaining considerable momentum, but there was something missing.

Yelp for Development

It came out as a total accident during a group meeting with USAID when we were creating the Global Development Commons, but the idea stuck: why not create a “Yelp” for development? Instead of trying to impose this measurement framework from on-high, why not crowdsource performance and impact ratings for everyone in the development space, from funders to implementers and everyone in between? The idea has its roots in basic human factors theory, and it was only when I came to EchoUser that I realized that it is such a powerful idea precisely because of its grounding in a tried-and-true methodology.

While the Yelp idea is compelling, my sense is that it’s a little grandiose. In the meantime, there have to be some intermediate steps that can test out some of the assumptions built into the idea. For example, who qualifies as an “end user” in development? Should we include everyone from the regional NGO to the local farmer? Who do we exclude and why? And my personal favorite: how to measure impact in a way that scales?

First things first: one of my main goals is to find a philanthropy/funding partner to help us test out our UME methodology in a development context. Then we can tackle the many other questions that will certainly arise. Any takers?

*Credit for the cartoon goes to the eminent Gary Larson and his Far Side series.

Rock on SVT.