ATTACHMENT_J-5.pdf

RFP 294-17-000003 ATTACHMENT J-5

ATTACHMENT J-5

HOW TO PREPARE AN EVALUATION REPORT

A. USAID Evaluation Report Requirements

An evaluation report must:

1. Identify the evaluation as either an impact or performance evaluation per the definitions in ADS 201.

2. Include an abstract of not more than 250 words briefly describing what was evaluated, evaluation questions, methods, and key findings or conclusions. The abstract should appear on its own page immediately after the evaluation report cover.

3. Include an Executive Summary 2–5 pages in length that summarizes key points (purpose and background, evaluation questions, methods, findings, and conclusions).

4. State the purpose of, audience for, and anticipated use(s) of the evaluation.

5. Describe the specific strategy, project, activity, or intervention to be evaluated including

(if available) award numbers, award dates, funding levels, and implementing partners.

6. Provide brief background information. This should include country and/or sector context;

specific problem or opportunity the intervention addresses; and the development hypothesis, theory of change, or simply how the intervention addresses the problem.

7. State the evaluation questions.

8. In an impact evaluation, state evaluations questions about measuring the change in specific outcomes attributable to a specific USAID intervention.

9. Describe the evaluation method(s) for data collection and analysis.

10. Describe limitations of the evaluation methodology.

11. In an impact evaluation, use specific experimental or quasi-experimental methods to answer impact evaluation questions.

12. Include evaluation findings and conclusions.

13. If recommendations are included, separate them from findings and conclusions.

14. Address all evaluation questions in the Statement of Work (SOW) or document approval by USAID for not addressing an evaluation question.

15. Include the following annexes:

• Evaluation SOW. If the SOW was revised over the course of the evaluation, the evaluation report should include the updated SOW as an Annex rather than the original SOW. The Contracting Officer’s Representative of the evaluation must agree upon, in writing, all modifications to the SOW, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline.

• A description of evaluation methods (if not described in full in the main body of the evaluation report).

• All data collection and analysis tools used, such as questionnaires, checklists, survey instruments, and discussion guides.

• All sources of information—properly identified and listed.

• Any “statements of difference” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team.

• Signed disclosures of conflicts of interest from evaluation team members.

• Summary information about evaluation team members, including qualifications, experience, and role on the team.

16. Include enough information on the cover of the evaluation report so that a reader can immediately understand that it is an evaluation and what was evaluated. The evaluation cover must:

• Include a title block in USAID light blue background color.

• Include the word “Evaluation” at the top of the title block and center the report title underneath that. The title should also include the word “evaluation.”

• Include the following statement across the bottom of the cover page: “This publication was produced at the request of the United States Agency for International Development.

It was prepared independently by [list authors and organizations involved in the preparation of the report].” For an internal evaluation team, use the following statement:

“This publication was produced at the request of [USAID/Mission or OU] and prepared by an internal evaluation team comprised of [list authors and affiliation].”

• Feature one high-quality photograph representative of the project being evaluated and include a brief caption on the inside front cover describing the image with photographer credit.

• State the month and year of report publication (e.g. when final and approved by USAID Operating Unit).

• State the individual authors of the report and identify the evaluation team leader.

As noted in ADS 201.3.5.17, evaluation reports must also conform to USAID branding requirements (see ADS 320, Branding and Marking) and comply with section 508 of the Rehabilitation Act (see ADS 302mak, USAID Implementation of Section 508 of the Rehabilitation Act of 1973). Prior to public dissemination, evaluation reports must be adjusted to remove information that falls under one of the “principled exceptions to the presumption in favor of openness” as described in ADS 201mae, Limitations to Disclosure and Exemptions to Public Dissemination of USAID Evaluation Reports.

B. Criteria to ensure quality of the Evaluation Report

Pursuant to 201.3.5.17, draft evaluation reports must undergo a peer review organized by the office managing the evaluation. The following criteria should serve as the basis against which the report is reviewed. To help ensure a high-quality evaluation report, these criteria must be included in the evaluation Statement of Work to communicate to evaluators USAID’s quality criteria.

• Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity.

• Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly.

• The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report.

• Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID.

• Evaluation methodology should be explained in detail and sources of information properly identified.

• Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).

• Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions.

• Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence.

• If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females.

• If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific.

[END OF ATTACHMENT J-5]