Attachment J.4 - DFID’s VfM Smart Guide.pdf

DFID’s Approach to Value for Money

March 2017 Finance and

Performance Dept.

This guidance aims to help our partners to better understand what we mean by Value for Money (VfM) in DFID, how we aim to maximise it across our work and what we expect from our partners.

The guidance:

• sets out what DFID means by VfM in the different areas of our work

• describes some key principles for understanding VfM in any context

• outlines some common VfM misconceptions

• discusses how to develop suitable VfM metrics

• set outs recommended VfM actions for partners

It also includes more detail on how DFID teams and our partners can ensure VfM in programme design and management.

What do we mean by Value for Money?

DFID has a duty to those living in extreme poverty and to the UK taxpayer to ensure that we do everything we can to maximise the VfM of our actions. This means making the best possible use of our resources to maximise our impact on poor people’s lives.

Understanding and improving VfM in international development is hard and complex. While we have made some progress and learnt many lessons over the past few years, we do not have all of the answers; there is always more than we can and should do.

VfM in DFID means that we maximise the impact of each pound spent to improve poor people’s lives

This means more than just the benefits of our work outweighing the costs. It means that in everything we do we try to maximise our impact1 on poor people’s lives, given the financial, political and human resources we have available.

Maximising VfM applies to all aspects of DFID’s work

At a strategic level, we work to improve the impact of all UK ODA and international development finance, to amplify the impact of our and others’ aid.

At a portfolio level, VfM means we aim to allocate our limited resources to maximise impact by doing the right things, in the right places, and in the right ways. We put in place strong corporate oversight to monitor and drive VfM.

At a programme level, VfM means we strive to design, procure, manage, and evaluate our interventions to maximise impact, given available resources. More detail on this is provided below.

At an administrative level, VfM means that the ways we work as an organisation maximise the impact that our people and resources can have. Our systems, cultures and behaviours empower staff to deliver more with less, whilst ensuring full accountability to the British taxpayer.

1 By impact, we mean long-term, transformational change.

March 2017

Across all of these levels, we invest in the central enablers to delivering VfM, such as transparency, data, scrutiny, performance-based funding mechanisms, and learning. A strong evidence base is essential for the provision of more effective development assistance, helping us to maximise VfM, learn lessons for the future and demonstrate impact.

Figure 2 at the end of this guide sets out diagrammatically what VfM means for DFID.

The key principles for understanding VfM

Maximising VfM means maximising our impact, given our resources. To do this we must:

- Work to understand and increase the benefits of our actions to poor people’s lives, and their sustainability.

- Work to understand and reduce the costs of our actions. This means costs to DFID in financial, human and political resource. It also means understanding potential costs to beneficiaries, or to other partners.

- Consider the benefits and costs of our work relative to what we expect would happen if DFID did not act – the counterfactual. This includes consideration of what other donors, NGOs and governments might do if DFID chose not to intervene.

- Assess our actions against credible alternatives – are there other, better ways to deliver the same intended result?

- Assess our actions strategically and consider the opportunity cost - with the same resources, could we deliver equivalent or greater impact on poverty elsewhere, or if we were to target different results?

Common VfM misconceptions

Costs: VfM doesn’t mean we only do the cheapest things. But it does mean that we have to get better at understanding our costs and at driving ourselves, our partners, and our contractors to deliver more for each pound we spend.

Benefits: we don’t just do the easiest things to measure. But we do have to get better at measuring impact. The more robust our evaluations and results, based on meaningful beneficiary feedback, the more confidently we can scale up high impact interventions and manage or close lower impact interventions, increasing our overall impact on poor people’s lives.

Risk: VfM does not necessarily mean low risk. Maximising the total development impact of DFID’s work will mean a balanced portfolio of programmes, with high-risk but potentially high-impact activities balanced by lower risk programming with more dependable development impact.

The core principle of risk management is to deliver our business objectives and in doing so maximise value for money.

Flexibility: VfM does not mean perfectly predicting costs and benefits before we begin work. Development can be messy, and we have an imperfect understanding of the conflict-

March 2017 affected and fragile places where many of those in extreme poverty live. Interventions in these environments often must adapt to deliver development impact, as we learn what works and what does not through doing. Delivering VfM requires ongoing validation of results, reassessment and adaptation, as we manage our programmes to maximise impact.

Speed: VfM does not always mean slower procurement for lower prices. In humanitarian crises or other very time-sensitive situations, the benefits of intervening rapidly can far outweigh the additional costs of procuring and mobilising rapidly. This does not mean that minimising costs and maximising benefits are not centrally important in these situations, but it does mean that the cost of time is much greater than in other development contexts.

Equity: DFID’s approach to VfM does not mean we should take the easy options, ignoring difficult to reach populations or problems which are difficult to tackle. But DFID faces very difficult decisions in deciding on how much of our resource we devote to humanitarian protection of those in most extreme and acute need, against how much which we devote to longer-term sustainable improvements in opportunities for those in deep poverty. These decisions cannot avoid being informed by our ability to have impact in different contexts, given our resources – in other words, the value for money of the options available to us.

Relevance: VfM is not something that applies only to programme design; it should drive decision making and management throughout the programme cycle and in relation to our strategic, portfolio and administrative activities.

VfM at the Programme Level

The Theory of Change

Our programmes often operate in uncertain and challenging environments over a number of years.

It is difficult to predict development impact with certainty at design stage. Given this uncertainty, each intervention should have a theory of change, which describes how change is assumed to come about through intervention in a prevailing situation.

The theory is usually laid out in a diagram showing the connections between interventions and outcomes – the causal pathways or results chains. But more than this, it makes clear that these causal pathways rest on a set of assumptions. And these assumptions have varying degrees of evidence to support them. It makes all three things explicit: causal pathways, assumptions, and evidence.

Creating a Theory of Change is beneficial because it:

1) Provides a framework to think logically through the assumed change pathways of an intervention, or design an intervention based on the current change pathways in a certain environment.

2) Prompts you to weigh the evidence behind each assumption of change, highlighting:

I. Evidence gaps, which can ideally be filled in design phase, or if none exists then;

II. Areas of weak evidence that need to be monitored and maybe evaluated

March 2017

3) Helps to identify potential blockages or risky pathways that need to be managed, the potential impact of those risks, and alternative change pathways that could act as a contingency plan.

4) Helps to identify opportunities - other partners/events/circumstances that contribute to an intended outcome, which could be strengthened rather than putting a new intervention in place, therefore offering better VFM.

5) Forms the basis of a results framework, often in the form of a logframe - helping to identify SMART2 outputs, outcomes, and impacts, in an open and transparent way.

A Theory of Change approach should be used in design, risk assessment, and to inform the monitoring and evaluation framework. Importantly, Theories of Change need to be periodically tested and if necessary revised, for example as new information comes to light. A well thought out theory of change forms the basis for a strong monitoring and evaluation approach, which drives VFM by enabling us to understand whether our projects are delivering the results we expect and, in some cases, by contributing to the global evidence base.

The Results Chain and the 4 Es

As explained above, part of setting out a theory of change for an intervention consists of identifying the result chain, which breaks down the steps between programme inputs and development impact.

Figure 1 below sets out the results chain, and what we mean by economy, efficiency, effectiveness, equity and cost-effectiveness. These consider the costs incurred by a programme to deliver results at different points on the chain. Maximising cost-effectiveness maximises overall VfM by maximising a programme’s impact on poor people’s lives, given the resources spent. Improving the economy, efficiency, effectiveness and equity of a programme are intermediate steps which help maximise cost-effectiveness.

2 Specific, Measurable, Agreed-upon, Realistic, Time-bound

March 2017

Figure 1 DFID’s Results Chain

------------------------------------------------------------Equity considerations ---------------------------------------------------- >

The ‘4 Es’ Economy - Are we (or our agents) buying inputs of the appropriate quality at the right price?

Efficiency - How well are we (or our agents) converting inputs into outputs? (‘Spending well’) Effectiveness - How well are the outputs produced by an intervention having the intended effect?

(‘Spending wisely’) Equity - How fairly are the benefits distributed? To what extent will we reach marginalised groups?

(“spending fairly”) Cost-effectiveness - What is the intervention’s ultimate impact on poverty reduction, relative to the inputs that we or our agents invest in it?

Principles - delivering VfM through the programme cycle

Maximising VfM in DFID’s programmes means that we design, procure, deliver, and close our interventions to maximise impact, given available resources. The text below summarises best practice in achieving VfM at each stage of the programme cycle.

At the design stage, achieving VfM means a robust business case which helps programme designers identify interventions which fit with our portfolio, and use evidence, experience and external engagement with the people we are aiming to serve to design options, management and procurement arrangements aimed at maximising the impact of each pound spent. Making clear our logic and assumptions through a testable theory of change should pave the way for effective procurement, programme management and evaluation. The expected VfM of a programme can be further improved by risk assessment and mitigation strategies, early market testing, beneficiary feedback and, where appropriate, pilot phases.

At the procurement/mobilisation stage, achieving VfM means minimising costs, given the quality and quantity of outputs required through robust and commercially savvy procurement; ensuring an appropriate balance of risk between DFID and our suppliers or delivery partners; ensuring that suppliers or delivery partners’ incentives are aligned with maximising development impact during

Input

Staff, raw materials, capital e.g. Vaccine and vaccination consumables

Process Outcome

Output Impact

Results delivered directly by DFID or our agents.

e.g. Children vaccinated

We exercise less direct control over outcomes than outputs.

e.g. Children less susceptible to major childhood diseases

Long-term transformational change

E.g. poverty reduced

The methods by which inputs are used e.g. Delivery logistics

Efficiency Economy Effectiveness

Cost-effectiveness

March 2017 programme delivery, and ensuring that the contract or agreement allows effective and suitably adaptive programme and contract management during delivery and at closure.

At the delivery stage, achieving VfM means a delivery plan, developed and maintained by the programme’s SRO, allowing delivery partners and DFID’s SRO to manage and adapt the programme to maintain or increase impact through delivery. Best practice would see managers efficiently monitor output indicators which accurately track the programme’s progress towards delivering impact, managing identified risk and validating results through engagement with key stakeholders. This helps managers respond promptly and proportionately if the programme goes off track, adapting the programme so that it remains VfM in light of unforeseen circumstances.

Equally, VfM is delivered where programme managers are able to act on opportunities to increase the impact of a programme, as new evidence and information comes to light.

Achieving VfM at the closure stage means that programme managers close a programme early if its expected additional impact no longer justifies the expected additional costs. Equally, programme managers extend programmes where the expected impact of a programme extension significantly exceeds expected extension costs, and where an extension has good strategic fit with DFID and other donor activity. All programmes need to ensure sustainability of outcomes at closure, something that needs to be built into project design and delivery.

Learning, evolving and adapting should occur at all stages of programme design and delivery.

Achieving VfM means learning from, not hiding, failure. It means proportionate monitoring and evaluation, including through annual reviews, which allows problems to be identified and programmes to be adapted promptly to maintain or enhance outcomes. It also means effective, proportionate knowledge management to allow lessons learned – either through formal knowledge or informal ‘know how’ - from a programme to be used in future programme design.

VFM Metrics

In addition to the logframe indicators, VFM metrics can play a valuable role in programme management – helping us to monitor key developments that impact the VFM of our programming and respond accordingly. VfM metrics can help complement the output scores tracked during the Annual Review.

Metrics are a means not an end; the objective of a metric is driving performance improvement. The aim is to be clear and transparent about the VfM of a project at the start, and to manage and improve that value through understanding and monitoring the main cost and impact drivers, ensuring the programme remains optimal use of DFID’s resources.

What is a VFM metric?

A metric is a measure: a value quantified against a benchmark at a point in time. A VfM metric can measure how much we get per pound spent, for instance:

• Input unit costs measure economy of key programme cost drivers; e.g. cost per kg of grain, or litre of fuel.

• Output unit costs measure efficiency; the costs of achieving one output, e.g.

the cost of a girl graduating, or getting an unemployed person into work.

• Outcome unit costs measure the cost of achieving an outcome, e.g. cost per death averted in health interventions.

• Cost-Benefit and Cost-Effectiveness Analyses measure cost effectiveness of alternative interventions.

March 2017

• Equity can be captured by disaggregating data across population groupings.

• Proxy metrics are also sometimes used in programmes or portfolios i.e.

metrics that drive VfM, but don’t directly link a cost to a result (e.g. leverage ratio of private sector finance, business failure rate in a given subsector of the economy).

Principles for VfM metrics

VFM metrics should be developed during project design; however where projects are already in implementation DFID will work alongside partners to develop suitable metrics. It is therefore important for partners to understand the general principles that we would expect good VfM metrics to follow. Most of these are dependent or overlap.

- Have a clear methodology: The metric should be clearly defined, to enable both consistent collection of data and calculation of metric value.

- Use benchmarking: On their own, VfM metrics do not answer the question ‘how well are we doing?’ A metric should have an appropriate benchmark against which performance can be assessed against to make a judgement on whether VfM is being realised or not. Benchmarks can come from other programmes, similar programmes in other countries or tracking of a value over time. Care should be taken in drawing comparisons: differences in country contexts often account for big differences in costs; the average cost of a past intervention does not tell you the marginal cost of achieving the next output; unit costs are rarely known with certainty in advance; and interventions can complement each other. The key is to examine the underlying data to understand the extent to which costs are comparable. Ultimately the valuable insights on VfM are drawn through trying to understand the differences and seeking to explain them.

- Keep things simple to interpret: Changes to the metric value should clearly represent an improvement or deterioration of VfM, or simple explanation should be feasible to explain what a metric is showing. It should easily guide decisions on what steps can be taken to improve value for money where it is not being realised.

- Be proportionate: Whatever metrics are used, the data required for them should not be excessively costly to gather and the metric should not be unreasonably resource intensive to calculate.

- Reflect main cost drivers: Economy measures should reflect the key cost drivers for the output or outcome being delivered. This involves understanding which different aspects of total cost contribute the most to the output or outcome being delivered and which of those are likely to change in a way that significantly affects the total cost of delivering the outcome.

Recommended VFM Actions for Partners

This section set outs the actions that partners are recommended to take in order to ensure VFM in every intervention. Details on how to take these actions forward should be discussed and agreed with the relevant team/office in DFID.

• Logframe & VFM metrics: all interventions should have a robust logframe with SMART output, outcome and impact indicators and targets. This should be agreed between DFID and partners/suppliers at the start of the project and should reflect the theory of change for

March 2017 the intervention. In case VFM metrics/indicators are to be captured outside of the logframe, details on data collection and reporting should also be agreed between DFID and partners.

• Understanding VFM: Partners/suppliers are expected to ensure that all of their staff understand the concept of VFM and how to apply it on a day-to-day basis.

• Reporting on VFM: Partner/supplier reports should include reference to VFM.

• Input costs and their drivers: As part of the effort to better understand the costs involved in delivering our programmes, partners/suppliers should be aware of their key input costs and cost drivers, including tracking these over time and, where possible, benchmarking input costs against relevant comparators (e.g. other projects, countries or market rates).

March 2017

Figure 2: How DFID systems aim to maximise VfM

DFID RESOURCES

IMPACT ON

POVERTY

LE

A

R N

IN

G

A D

A P

TI

N

G

Right things

Right places

Right ways

Portfolio priorities

SYSTEM

S

C

PA

B

ILITIES

D

M

IN

ISTR

A

TIV

E

C O

O R

D

IN

A

TIO

N

PA

R

TN

ER

SH

IP

S

STR

TEG

IC

Tr a n sp a re n cy

VfM in DFID means that we try to maximise our impact on poor people’s lives, given the financial, political and human resources we have available.

In flu en cin g o th er d o n o rs a n d p a rtn er g o vern m en ts