Helping NGOs do more with their money

We're moving to Humentum.org soon.

Payment by results

The National Audit Office (NAO) issued its review of the UK Government, including DFID's, use of Payment By Results (PbR) on 19 June 2015.  This followed DFID's publication of its Evaluation framework for PbR the day before.  As the Mango Research Paper into PBR commented previously,  it's early days for DFID's learning journey into PbR as they only published their PbR strategy in June 2014.

I share below 5 key insights I drew from the careful words in the NAO report.

INSIGHT 1 - There is still no convincing evidence that PbR drives good value for money, or innovation, yet......

“16 While supporters argue that by its nature PbR offers value for money, PbR contracts are hard to get right, which makes them risky and costly for commissioners. If PbR can deliver the benefits its supporters claim – such as innovative solutions to intractable problems – then the increased cost and risk may be justified, but this requires credible evidence. Without such evidence, commissioners may be using PbR in circumstances to which it is ill-suited, with a consequent negative impact on value for money.” (From the Conclusion on page 9)

INSIGHT 2 - Is the NAO worried that DFID is flying a little blind?

“13 It is essential that commissioners establish performance expectations at the start of a scheme, taking into account baseline performance and non-intervention rates. Commissioners should aim to define attainable but stretching performance expectations for providers that are above the non-intervention rate (the level of performance that would occur without intervention). This avoids payment for performance that would have occurred anyway. Commissioners need good data on baseline performance, and to carry out robust modelling of likely future performance and sensitivity testing of any assumptions underpinning estimates. Where the commissioner is using PbR for a new service, such as international aid, the lack of historic, comparable data can make this especially challenging (paragraphs 3.5 to 3.7).” (page 7, my italics)

3.7 Determining baseline and non-intervention rates can be challenging and often requires assumptions and estimates to be made. DFID faces a particular challenge in setting expected performance for its PbR contracts because in many international aid settings the relevant data are unavailable. DFID’s PbR project on rural water supply and sanitation in Tanzania estimated that using PbR would lead to a 10% increase in performance. However, the project’s business case acknowledged the limited evidence to support the estimate because outcome-based PbR had not been tried before in the sector.” (page 23)

INSIGHT 3 - The NAO ask big questions about whether PbR can deal with attribution problems in the complex aid & development environment.

Figure 6 “Some PbR schemes (particularly DFID’s) involve complex delivery chains, making it harder to ensure effective accountability for frontline providers.” (page 20)

INSIGHT 4 - It is interesting that the NAO passes no comment on DFID’s ‘learning by doing’ strategy

4.16 “None of DFID’s projects have yet reached the final evaluation stage but DFID has developed a framework for evaluating PbR schemes, which recommends evaluations include consideration of schemes’ impact and value for money.” (page 38)

INSIGHT 5 - I wonder where the risk really lies on the £2.2bn DFID has already committed to PbR projects - with the learner or the doer?

DFID’s PbR portfolio (from page 4)

No of PbR projects

19

Budget

approximately £2.2 billion

Duration

projects run for between 3 and 12 years

PbR proportion of contract:

varies between 9% and 100%

Providers

Various governments, private sector suppliers and non-governmental organisations

Aims

A range of development issues including water and sanitation, education and health.


Download the Mango research paper into PbR here:

 

 

 

 

 

 

 

 

 

 

 

 

 

I'd be very interested to hear what you think.  You can tweet me at @tim4Mango.

Tim Boyes-Watson, Director, Mango

 

Close