Acknowledgements: with thanks to contributors Jennifer Armitage and Adesoji Ologun

About High Impact Clients

Kiran, a 22-year-old woman is 5 months pregnant and lives in Balochistan province in Pakistan. The province is the least populated with approximately 5% of the total population and is mostly isolated because if its mountainous and arid terrain^[1]. She lives on less than $2 per day and she has mobility difficulties. She recently commenced ante-natal care clinic but will not be attending clinic regularly because she is unable to physically access the ante-natal clinic because of lack of adequate facilities.

Kiran is an example of a High Impact Client (HIC) that will benefit from health programmes targeted at eradicating poverty in all its forms, end discrimination and exclusion and reduce inequalities and vulnerabilities that leave people behind and undermine the potential of individuals and of humanity as a whole^[2].

Purpose of the Equity Costing Analysis

LAMP therefore set out on this journey to estimate the cost of leaving no one behind on a donor-funded multi-country programme implemented in East and Central Africa as well as South-East Asia.

The purpose of the Equity Costing Analysis was to estimate the cost of reaching the high impact clients targeted by the donor-funded programme so that the information can be used to improve decision making on future programmes.

Estimating the Unit Cost of reaching High Impact Clients

While it can be simple to estimate the unit cost of reaching HICs by estimating the number of beneficiaries of the interventions from coverage figures available on the programme and the cost of reaching them, this is not always the case. This is because often, in addition to reaching HICs, programmes also target other persons in the general population, so it was important for the analysis to identify the cost of reaching these HICs specifically as well as the number of actual beneficiaries. As a result, the study estimated the total cost of reaching HICs and non-HICs, the cost of implementing activities specifically targeted at HICs only, known as additional cost of reaching HICs, and the cost of reaching the general population (non-HICs).

The analysis was based on existing data available within the programme in order to minimise the burden of data collection on the teams who were busy running the programme. Cost data was collected from the programme budget, workplans and activity reports.

The analysis set out to estimate the unit cost of reaching HICs based on the assumption that implementing partners would be able to identify the additional activities and cost incurred in reaching HICs. However, there was no clear distinction between the cost of reaching HICs and non-HICs for activities implemented by the programme because there was inadequate data on the profile of the general population and HICs in the target countries. Furthermore, the cost of reaching HICs was embedded in the overall activity implementation cost so was not captured separately by the programme.

Challenges with estimating the additional cost of reaching HICs and the unit cost of reaching HICs

• Identification of high impact clients: The categories of HICs targeted in the countries by the implementing partners was not properly defined because the same client can be under 20, poor and have a disability. Also, HICs and non-HICs could not be delineated because the programme reached HICs using activities targeted at non-HICs.
• Identification of interventions targeted at the HICs: IPs retrospectively identified the activities implemented to specifically target HICs so it may be possible to have either under-reported or over-reported these activities due to recall bias.
• Estimation of additional cost of reaching HICs: This became a challenge because the additional costs estimated included cost of programmes that may or may not have targeted both HICs and non-HICs which resulted in over or under-estimation of the ‘additional’ cost of reaching HICs identified in the study.
• Quality of cost data available from the programme: The financial system of the programme was not set up to collect the data for the analysis because it was not conceptualized at the onset of the programme.
• Limitations related to outcome measures: The outcome indicators only considered the number of times the beneficiary encounters the system / number of beneficiaries using the system(users), this does not consider the quality of services delivered.

Recommendations for future programming

Future programmes can capture HIC data better by using measures such as wealth quintile available in Demographic Health Surveys and poverty assessment tools that can guide programming for the poor. Poverty and disability registers^[3] can be used to track programme beneficiaries to collect information on social measures of equity while geospatial analysis can be used to identify geographic remoteness and assess differences across areas such as urban/rural, humanitarian, or natural disaster settings.

A programme should include a framework for operationalising equity during programme design. This includes identifying groups, individuals or communities that face inequities, clearly defining activities to address these disparities, developing data collection tools to collect the information related to these activities, ensuring financial management system is optimized to properly categorizes costs related to reaching vulnerable populations and evaluates the results related to the interventions implemented.

Conclusion

People like Kiran exist in different implementation areas and they will remain the focus of donor and nationally funded programmes.  As a result, these programmes should continue to explore ways to measure and demonstrate increasing access to services for poor, people with disabilities, people living in hard-to-reach areas and adolescents. In addition, future donor-funded programmes could explore further analysis using methodologies like distributional cost-effectiveness analysis (DCEA) which offers a framework for incorporating equity concerns into cost effectiveness analysis.

[1] Special Development Scheme for the Uplift Of 20 Poorest Districts In Pakistan  https://www.pc.gov.pk/web/poorestdistricts

[2] Universal Values Principle Two: Leave No One Behind https://unsdg.un.org/2030-agenda/universal-values/leave-no-one-behind#:~:text=It%20represents%20the%20unequivocal%20commitment,of%20humanity%20as%20a%20whole.

[1] Missing Billion Initiative and Clinton Health Access Initiative, Reimagining health systems that expect, accept and connect 1 billion people with disabilities, MBI and CHAI, 2022.

The Lagos experience  

In 2017, LAMP conducted one of the first social return on investment (SROI) studies on the digitisation of health systems. In partnership with E4A-MamaYe, LAMP sought to assess the value of the early stages of the digitisation of the maternal, perinatal death surveillance and review (MPDSR) system in Lagos, Nigeria. With approximately 555 maternal deaths per 100,000 live births per year in Lagos, the MPDSR system is vital to helping facilities and state health authorities understand the underlying factors contributing to maternal death and act to prevent similar deaths happening in the future. It is critical to ensure that digitisation supports this and the SROI study aimed to provide insights into how the process was working and make recommendations for future roll-out. 

Why Social Return on Investment?

SROI is a framework for understanding, measuring, and reporting the social, economic and environmental value created by an intervention, programme, policy or organisation. Basically, it measures how worthwhile an investment is from the perspective of stakeholders. It accounts for direct and indirect costs and provides a method for monetary valuation of both positive and negative health and non-health outcomes. It can be used to retrospectively measure outcomes that have already occurred (evaluative-type) or can prospectively predict how much value will be generated if the intervention meets its intended outcomes (forecast-type). There has been a lot of interest in the development community in using SROI, as it has the capacity to capture broader outcomes of public health interventions and assess their value-for-money.

In Lagos, where maternal and neonatal death surveillance is going digital, stakeholders wanted to find out whether this would represent a true return on investment SROI measures how worthwhile an investment is from the perspective of stakeholders Insights generated from the research helped to improve roll-out of the digitised system

Using SROI in Lagos  

LAMP worked closely with the implementers to develop a social return on investment methodology to align with the intervention’s theory of change. The research involved engaging with stakeholders at the Lagos State Ministry of Health and running a series of focus group discussions with clinicians at maternity hospitals across the State, as well as collecting existing costs and outcomes data of the digitised platform.

Findings and feedback on use of SROI

Using SROI, LAMP captured the anticipated and unanticipated benefits and challenges experienced by stakeholders during the early phases of digitisation. These included improved accuracy of data and simplified workloads as a result of transferring data entry into digital systems. Medical personnel felt the forced data-entry in the digital system made them better clinicians by helping them record a more thorough patient history. However, there were also challenges in digitisation including: stakeholders fear about losing the tablets, access to and functionality of the online data platform. These insights provided a holistic view on the progress of the digitisation and enabled E4A-MamaYe to assess whether going digital would represent a true return on investment. The findings and insights enabled E4A-MamaYe programme staff, in conjunction with the Lagos State Ministry of Health, to tailor their digital strategy which is currently mid-roll out.

How can SROI work for you?

Overall, the SROI methodology allowed for the capture of broader outcomes, positive and negative, that had not been previously characterised or associated with digitisation of MPDSR. It enabled us to provide a monetary value to typically non-monetary outcomes and include these in assessments of value for money. Unlike more conventional economic evaluation methods such as cost-effective and cost-utility analyses (CEA & CUA), in which outcomes are more unidimensional – an SROI approach can give a more reflective and honest assessment of the intervention. From our experience, this is the unique perspective that SROI can offer for your work.