Ideen evaluieren statt Projekte

Wenn es um die Erfolgsbewertung (Evaluierung) von Entwcklungszusammenarbeit (EZ) geht, ist die Frage danach, "was funktioniert" nach Meinung von Chris Blattman meist falsch gestellt. Why “what works?” is the wrong question: Evaluating ideas not programs. Besser wäre es zunächst zu fragen: Funktioniert die Welt tatsächlich so, wie wir annehmen und wie unser Projektdesign das voraussetzt?

What we want is a reasonably accurate model of the world: why people or communities or institutions behave the way they do, and how they will respond to an incentive, or a constraint relieved.

Blattman begründet seine Einsichten mit seinen eigenen Erfahrungen in Liberia, wo es ihm nur unglückliche(?) Umstände ermöglichten, auch die einem Projekt zur Wiedereingliederung von Ex-Kombattanten zugrunde liegenden Annahmen zu testen.

These 1 und 2 wurde bestätigt: Es braucht nicht nur Ausbildung (1) sondern auch Startkapital (2) für ökonomischen Erfolg.

The men who expressed interest in raising vegetables received their seeds and tools. But the men who said they were interested in raising animals waited more than 18 months for their chicks and piglets to arrive. (...) Not surprisingly, the vegetable men had higher farm profits than the men still waiting for their animals to arrive.

Bei These 3 war die Sache komplizierter: Denn Erfolge in dem neuen Leben machen die Wiederaufnahme illegaler Tätigkeiten nicht unbedingt unwahrscheinlicher.

Then something even more tragic and more important happened: a war broke out in neighboring Cote d’Ivoire. Ex-commanders around the country began to hold meetings and recruit. (...)
An interesting thing about who did and didn’t say they got involved in recruitment (and an interesting thing about who did and didn’t return to the illegal mines): the vegetable farmers were a little less likely than the control group to express interest in recruitment. They had greater incomes after all. But the men waiting for their piglets and chicks were the least likely to express interest or plans for mercenary work. Apparently, a promise of $100 worth of animals was worth more than the $500 for getting onto the truck to Cote d’Ivoire. Or so they told us.

Unabhängig von diesen Ergebnissen untersuchte Blattman in einem anderen Projekt ebenfalls in Liberia die "kaum glaubliche Prämisse", ob eine nur acht Wochen dauernde Gruppen- und Verhaltenstherapie vielleicht auch helfen könne, Kriminalität und Gewalt zu mindern.

So we put that to the test. And because we wanted to test an alternative approach, we also evaluated a cash transfer to the same men. So men could receive one, both, or none of the interventions.

Mit dem überaus positiven Ergebnis hatte niemand gerechnet:

We found that the therapy was incredibly powerful, drastically reducing crime and violence. And we found it was even more powerful (and lasting) when an economic intervention provided petty criminals with even a short period of time in which to practice their new skills of self control and new self image. Crime and aggression fell by half.

Der nächste Schritt sei jetzt, Fragestellungen und Evaluierungsdesigns zu entwickeln, die es erlauben, festzustellen, ob bestimmte Annahmen generell gültig sind. Dazu müsse man z.B. dieselben Fragestellungen in verschiedenen Ländern testen. Das würde auch helfen, die Theorien zu verbessern.

Take the therapy example. The next logical step is cognitive behavioral therapy trials in more places. There are enough examples from the US and Liberia that cognitive behavioral is ripe for piloting and testing at scale in several countries, ideally testing implementation approaches in each place: longer or shorter, with or without economic assistance, and so forth.

Natürlich liefert Blattman Ideen für weitere Untersuchungen:

- The right combination of skills and capital to raise incomes is also ripe for testing in many countries.
- When people do become wealthier and more employed, are they less willing to fight?
-  Are their votes harder to buy? Are they less tolerant of a corrupt regime? Maybe anti-poverty programs have a hidden impact on governance that we are not taking into account. 

Die Liste möglicher Fragestellungen ist sicher fast endlos, aber es lohnt sich mit der Arbeit anzufangen. Denn sonst entsteht fundiertes Wissen über die Zusammenhänge wohl immer nur in kleinen Stückchen und oft nur zufällig.

In structuring themselves to be country driven, development agencies have created a collective action problem. The public good of knowledge is still getting created, but it’s slow, piecemeal, and sometimes accidental.
I would make sure they more seldom evaluate programs, and more often investigate assumptions and ideas. And never just any ideas, but the legs that prop up enormous things. If strengthened, these legs can have enormous impact. And if a weak leg is kicked out, then the money used for the kicking probably could not have been better spent.