Annex_G_-_Evaluation_Design_Report_Outline_Template.pdf

MCC Independent Evaluation Design Report Outline Template

i. List of Acronyms

ii. Table of Contents

LIST OF ACRONYMS

TABLE OF CONTENTS

1. INTRODUCTION

1.1. Background

1.1.1. Country context

1.1.2. Objectives of this report

2. OVERVIEW OF THE COMPACT AND THE INTERVENTIONS EVALUATED

2.1. Overview of the project and implementation plan

2.1.1. Program participants (who is being targeted and why)

2.1.2. Geographic coverage (where it is being implemented and why)

2.2. Key program indicators and how they lead to expected outcomes. (Here we will summarize the program logic rather than discuss its merits since the program logic assessment is coming in the assessment report.)

2.3. Link to ERR and Beneficiary Analysis

3. LITERATURE REVIEW

3.1. Summary of the existing evidence

3.2. Gaps in literature

4. EVALUATION DESIGN

4.1. Policy relevance of the evaluation

4.2. Overview of an impact evaluation that includes a quantitative impact analysis and qualitative implementation analysis

4.3. Evaluation questions

4.4. Key Outcomes

4.5. Methodology/ Impact evaluation design

4.6. Study Sample (to include required sample size, sample frame, process of selection of treatment and control groups, stratification, design omission in the sample, level of representation, strategy for absent respondents and dropouts)

4.7. Analysis Plan

4.8. Timeframe of exposure. Will include justification for survey timing relating back to program logic and when impacts are anticipated to take effect.

4.9. Limitations and Challenges. Will include strategies to minimize problems with randomization compliance and strategies to deal with deviation from assignment status.

5. DATA SOURCES AND OUTCOME DEFINITIONS

5.1. Data collection plans. (We are thinking it would be good to discuss survey rounds and when different types of data collection would take place.)

5.2. Proposal to measure implementation fidelity

5.3. Data needs

5.3.1.1. Data sources for quantitative

5.3.1.2. Data sources for qualitative

Commented [SJR(1]: If the discussion in Program Logic sufficiently covers this, no need to repeat. However, if the evaluation is only looking at certain outcomes, not all, then please make sure that is clarified either above in program logic or in this section.

6. ADMINSTRATIVE

6.1. Summary of IRB requirements and clearances

6.2. Preparing data files for access, privacy and documentation

6.3. Dissemination Plan

6.4. Evaluation Team Roles and Responsibilities

6.5. evaluation timeline

6.6. reporting schedule

7. REFERENCES

8. ANNEXES

8.1. Stakeholder comments and Evaluator Responses

8.2. Evaluation Budget. Given sensitivities around future procurements, the budget should not be a public document. However, the evaluator and MCC M&E should have a clearly defined budget for the evaluation, including planned evaluator and data collection costs which should be tracked over time with actual evaluator and data collection costs.