Independent Evaluation of Development Impact Bonds

We are currently providing inputs to the Independent Evaluation of the Development Impact Bonds (DIBs Pilot Programme).  The purpose of the evaluation is to generate learnings and recommendations on the use of DIBs as an instrument for aid delivery, by using the experience of the DFID DIBs pilot programme to generate learning to inform DFID’s future policy aiming to make the most effective use of DIBs. The evaluation will also help DFID and pilot project partners evaluate whether the tools they are developing are useful, scalable and replicable.

Mobilising for Development (M4D) – Value for Money Analysis

Mobilising for Development (M4D) was a six-year local governance programme in three northern states of Nigeria funded by the UK Department for International Development (DFID). The programme aimed to “support local and state level policy makers and service providersto be more responsive and accountable to citizens’ better articulated demands and entitlements, especially adolescent girls (AG) and people with disability (PWD)”.

LAMP conducted a value for money analysis at the end of the programme. The analysis assessed the programme’s VFM based on the 4 Es framework (Economy, Efficiency, Effectiveness and Equity) drawing on programme data and combining this with benchmarking and comparative data. The analysis also assessed whether the improvements achieved by the programme, in terms of service delivery, accountability and inclusion, offered good value for money and in particular whether there was evidence of these improvements being catalytic and sustainable. It also documented lessons learned on measuring and demonstrating VFM in a local governance programme.

Integrated Reproductive, Maternal and Newborn Health (IRMNH) – VFM Benchmarking and CEA

The IRMNH program is a five-year programme which supported the use of a nationwide comprehensive package of reproductive, maternal and newborn health care services across Sierra Leone. The programme is innovative in that it brings together major implementing partners in the health sector to support the Ministry of Health and Sanitation under one umbrella programme.

LAMP conducted a regional Value for Money Benchmarking Study and Cost Effectiveness Analysis of the IRMNH program. The VfM benchmarking was a desk-based review of the value for money performance indicators of the program together with other similar programmes in Sierra Leone and the West African region using available programme documents. Also, using the Spectrum Policy Modelling System Software and programme data made available, the Cost Effectiveness Analysis of the program was conducted to estimate the potential impact of the IRMNH intervention.

The results of the studies provided programme implementers with information that can be used for accountability and advocacy purposes. In addition, it provided a better understanding of the IRMNH programme’s impact and information to inform future design.

Strengthening Referral Pathways from the Street to School – Endline Evaluation

Endline Evaluation of the EU/NSA project – Strengthening Referral Pathways from the Street to School – Supporting Non State Actors to Protect and Empower Extremely Vulnerable Children and Youth in Sierra Leone

The GOAL Ireland in collaboration with four partners – St Georges Foundation, Sierra Leone Aid Volunteers, Integrated Children and Youth Initiative and Ben Hirsh implemented the EU/NSA project. The aim of the project was to build the capacities of Non State Actors, Local Authorities, and Government to deliver quality basic social services to EVCY by strengthening referral pathways from the Streets to Schools in Western Urban Area and Kenema districts in Sierra Leone.

GOAL commissioned Institute of Development (IfD) to conduct the evaluation of the EU/NSA project based on Development Assistance Committee (DAC) criteria which assessed the relevance, effectiveness, efficiency, sustainability and impact of the project. LAMP partnered with IfD to deliver the efficiency aspect of the evaluation due to its expertise in value for money analysis. Using programme cost and outcome data made available, cost analysis of the programme was conducted and cost per specific programme output estimated. In addition, findings from the qualitative aspect of the evaluation was used to inform the quantitative data analysis and results.

This evaluation will help GOAL improve its future programming through capturing the successes as well as the challenges of this programme and lessons learned.

Strengthening Research and Knowledge Systems VFM analysis

The Strengthening Research and Knowledge Systems (SRKS) programme, was a five-year, £10.2m programme jointly funded by DFID and the Swedish International Development Cooperation Agency (Sida) and implemented by INASP between 2013 and 2018. The programme aimed to increase sustainable (a) access to and (b) production of research literature in particular low income countries. The expected impact was that research knowledge contributes to economic and social development. The programme was being delivered in 24 countries at once, with focus on Tanzania, Sri Lanka, Ghana and Kenya.

 

LAMP provided inputs to the evaluation including a value for money assessment of the programme.

MamaYe – Social Return on Investment analysis

Started in 2012, E4A-MamaYe has used a combination of evidence and advocacy to drive accountability for maternal and newborn outcomes. E4A-MamaYe acts as a catalyst for action, using evidence strategically to generate political commitment, strengthen accountability and improve planning and decision making at sub-national and national levels.

In Nigeria, E4A-MamaYe supports the implementation of Maternal Perinatal Death Surveillance and Response (MPDSR) in three states and supports accountability mechanisms to ensure the use of findings from MPDSRs to guide advocacy and drive action to improve quality of care.

LAMP designed a study protocol for a social return on investment analysis on the digitisation of the Maternal Perinatal Death Surveillance and Response system in Nigeria.

Scaling Up Family Planning Programme VFM analysis

The Scaling Up Family Planning Programme in Zambia has been running since 2012. It was successful in giving 300,000 women and girls access to family planning (50% more than the original target), with poor rural women making up more than 70% of beneficiaries. A costed extension to the end of 2018 was agreed in 2016. The overall objective of the programme is to improve reproductive health through providing women with access to family planning. The focus of the costed extension (SUFP II) is to enable the adaptation of the SUFP model and transition to full government management of the programme at all levels, based on lessons learned to date. This means the project has shifted from a service delivery programme, whereby SUFP directly conducted activities to increase access to FP services, to a capacity development and systems strengthening programme.

LAMP was contracted and worked closely with the programme team to:

  • Apply SUFP VFM framework to a reporting system, which aligns with existing processes as far as possible, for example, annual work plans, and operational performance indicators from M&E logframe and financial inputs from financial management system.
  • Generate, verify and analyse year-11 VFM indicators and results.
  • Review costs per output indicators, taking into consideration long term discounted results.
  • Generate a VFM report which will cover the first year of SUFP II.
  • Compare results to baseline surveys taken from SUFP phase 1, year 4 where applicable and appropriate.

 

Beyond that, LAMP worked closely with the programme team to:

  • Prepare a VfM report for the first year of the 2nd phase of the programme
  • Revise and update the VfM framework reflecting the evolution of the programme from service delivery to systems strengthening
  • Make recommendations for further VfM improvements to the programme

Nepal Health System Support Programme 3 – VFM analysis

Options has been supporting the Government of Nepal (GoN) to improve its health services, for over 20 years. GoN has been committed to improving the health status of its citizens and made impressive gains despite conflict and other difficulties. The Ministry of Health is currently implementing the Nepal Health System Strategy (NHSS) 2015-2020. Between May 2017 and December 2020, a consortium led by Options, with HERD, OPM, and Miyamoto will be supporting the implementation of this strategy, through the Nepal Health System Support Programme 3 (NHSSP 3), funded by DFID.

LAMP developed a flexible and responsive VfM framework which will both demonstrate and improve VfM of the programme.

UNICEF prevention of adolescent suicide – Economic Evaluation

The WHO defines health as ‘a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity’; nevertheless, mental health is too often neglected. Prevention of mental health disorders focus on three areas of action:  promoting a healthy sense of well-being , identifiying  those at risk of mental ill health, and lastly treatment for those who need it.

At the global level, Kazakhstan was ranked as the 9th and 4th for suicide rates in children aged five to14 and youth between 15-29 respectively (WHO 2014). In response to this, in early 2015 the ministries of Health, Education and Internal Affairs and UNICEF developed an intersectoral programme on the prevention of suicides among adolescents which was initiated as a two-year pilot in Kyzylorda region (one of the 16 provinces of Kazakhstan).

LAMP provided support to the evaluation of this programme, undertaking the economic evaluation component.  We worked remotely with the evaluation team conducting the fieldwork in Kazakhstan to develop an economic evaluation including a cost analysis of the programme.

SROI evaluation of eMPDSR Lagos, Nigeria

Evidence for action (E4A) seeks to improve maternal and newborn survival through a combined focus on evidence, advocacy and accountability in Nigeria and two sub-Saharan African countries. One of its interventions is the strengthening of Maternal and Perinatal Death and Surveillance Response (MPDSR) in Lagos State, Nigeria. The MPDSR is a qualitative, in-depth investigation into the causes of, and circumstances surrounding maternal or perinatal deaths that occur in health facilities. In May 2017, this paper-based process was digitized to improve the quality of maternal and newborn health services and outcomes in Lagos State.

E4A commissioned LAMP to conduct a Social Return On Investment (SROI) evaluation of the digitization of the Maternal Perinatal Death Surveillance and Response (eMPDSR) system in Nigeria. SROI is a framework for measuring and accounting for the broader impact of an intervention. It involved evaluating the digitization of the MPDSR based on opinions of key stakeholders and analysis of expected outcomes. During the study, stakeholders were identified, outcomes were mapped, these outcomes were evidenced and given a value, the impact of the intervention was established and the SROI ratio was calculated. The SROI methodology was used because it captures the range of opinion from different stakeholders that have been involved in the implementation of the digitization process.