SOL-617-15-000011_Final.pdf

LARA P&IE

SOL-617-15-000011

SOLICITATION, OFFER AND AWARD

1. THIS CONTRACT IS A RATED

ORDER UNDER DPAS (15 CFR 700)

RATING

N/A

PAGE OF PAGES

1 76

2. CONTRACT NUMBER 3. SOLICITATION NUMBER

4. TYPE OF SOLICITATION

SEALED BID (IFB)

X NEGOTIATED (RFP)

5. DATE ISSUED

06/19/2015

6. REQUISITION/PURCHASE NUMBER

REQ-617-15-000030

7. ISSUED BY

COD

8. ADDRESS OFFER TO (If other than Item 7)

OFFICE OF ACQUISITION AND ASSISTANCE USAID/UGANDA

PLOT 1577 GGABA ROAD KAMPALA,

UGANDA

Email: KampalaUSAIDSolicita@USAID.gov

OFFICE OF ACQUISITION AND ASSISTANCE

USAID/UGANDA

2190 Kampala Pl Washington, DC 20521-2190

NOTE: In sealed bid solicitations "offer" and "offeror" mean "bid" and "bidder".

SOLICITATION

9. Sealed offers in original and Electronic only copies for furnishing the supplies or services in the Schedule will be received at the place specified in Item 8, or if hand carried, in the depository located in N/A until 1600 EDT localtime(Hour)

08/27/2015 (Date)

CAUTION: LATE Submissions, Modifications, and Withdrawals: See Section L, Provision No. 52.214-7 or 52.215-1. All offers are subject to all terms and conditions contained in this solicitation.

10. FOR

INFORMATI

ON

CALL:

A. NAME

Jacqueline Mirembe

B. TELEPHONE (NO COLLECT CALLS) C. E-MAIL ADDRESS

jmirembe@usaid.gov AREA CODE 256-414

NUMBER

306-001

EXT

11. TABLE OF CONTENTS

(X) SEC. DESCRIPTION PAGE(S) (X) SEC. DESCRIPTION PAGE(S)

PART I - THE SCHEDULE PART II - CONTRACT CLAUSES

X A SOLICITATION/CONTRACT FORM 5 X I CONTRACT CLAUSES 47

X B SUPPLIES OR SERVICES AND PRICES/COSTS 9 PART III - LIST OF DOCUMENTS, EXHIBITS AND OTHER ATTACH.

X C DESCRIPTION/SPECS./WORK STATEMENT 10 X J LIST OF ATTACHMENTS 58

X D PACKAGING AND MARKING 22 PART IV - REPRESENTATIONS AND INSTRUCTIONS

X E INSPECTION ANDACCEPTANCE 24 X K REPRESENTATIONS,

CERTIFICATIONS AND OTHER

STATEMENTS OF OFFERORS

X F DELIVERIESORPERFORMANCE 25

X G CONTRACT ADMINISTRATION DATA 30 X L INSTRS., CONDS., AND NOTICES TO OFFERORS 69

X H SPECIAL CONTRACT REQUIREMENTS 32 X M EVALUATION FACTORS FOR AWARD 75

OFFER (Must be fully completed by offeror)

NOTE: Item 12 does not apply if the solicitation includes the provisions at 52.214-16, Minimum Bid Acceptance Period.

12. In compliance with the above, the undersigned agrees, if this offer is accepted within calendar days (60 calendar days unless a different period is inserted by the offeror) from the date for receipt of offers specified above, to furnish any or all items upon which prices are offered at the price set opposite each item, delivered at the designated point(s), within the time specified in the schedule.

AWARD (To be completed by government)

19. ACCEPTED AS TO ITEMS NUMBERED 20. AMOUNT 21. ACCOUNTING AND APPROPRIATION

22. AUTHORITY FOR USING OTHER THAN FULL AND OPEN COMPETITION:

10 U.S.C. 2304 (c) ( ) 41 U.S.C. 253 (c) ( )

23. SUBMIT INVOICES TO ADDRESS SHOWN IN

(4 copies unless otherwise specified)

ITEM

24. ADMINISTERED BY (If other than Item 7) CODE 25. PAYMENT WILL BE MADE BY CODE USAID/Uganda Financial Management Office

26. NAME OF CONTRACTING OFFICER (Type or print)

Jennifer L Crow-Yang

27. UNITED STATES OF AMERICA

28. AWARD DATE

13. DISCOUNT FOR PROMPT PAYMENT

(See Section I, Clause No. 52.232.8)

10 CALENDAR DAYS (%) 20 CALENDAR DAYS (%) 30 CALENDAR DAYS (%) CALENDAR DAYS (%)

14. ACKNOWLEDGEMENT OF AMENDMENTS

(The offeror acknowledges receipt of amendments to the SOLICITATION for offerors and related documents numbered and dated):

AMENDMENT NO. DATE AMENDMENT NO. DATE

15A. NAME

AND

ADDRESS

OF

OFFEROR

CODE FACILITY 16. NAME AND TITLE OF PERSON AUTHORIZED TO SIGN OFFER

(Type or print)

15B. TELEPHONE NUMBER 15C. CHECK IF REMITTANCE ADDRESS

IS DIFFERENT FROM ABOVE - ENTER

SUCH ADDRESS IN SCHEDULE.

17. SIGNATURE 18. OFFER DATE

AREA CODE NUMBER EXT.

mailto:KampalaUSAIDSolicita@USAID.gov mailto:jbanco@usaid.gov

PART 1 – SCHEDULE

SECTION B - SUPPLIES OR SERVICES AND PRICE/COSTS

B.2 CONTRACT TYPE

B.3 TOTAL PRICE AND OBLIGATED AMOUNT

SECTION C – STATEMENT OF WORK

C.1 OVERVIEW

C.2 BACKGROUND

C.3. EVALUATION RATIONALE

C.4 EVALUATION DESIGN AND METHODOLOGY

C.5 EVALUATION PRODUCTS

C.6 GUIDING PRINCIPLES AND USAID POLICY

SECTION D- PACKAGING AND MARKING

D.1 AIDAR 752.7009 MARKING (JAN 1993)

D.2 BRANDING POLICY

D.3 BRANDING AND MARKING STRATEGY

SECTION E- INSPECTION AND ACCEPTANCE

E.1 NOTICE LISTING CONTRACT CLAUSES INCORPORATED BY REFERENCE

E.2 INSPECTION AND ACCEPTANCE

E.3 QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

SECTION F - DELIVERIES OR PERFORMANCE

F.1 NOTICE LISTING CONTRACT CLAUSES INCORPORATED BY REFERENCE

F.2 PERIOD OF PERFORMANCE

F.3 PLACE OF PERFORMANCE

F.4 REPORTS AND DELIVERABLES OR OUTPUTS

F.5 SYNOPSIS OF CONTRACT REPORTS/PLANS

SECTION G - CONTRACT ADMINISTRATION DATA

G.1 AIDAR 752.7003 DOCUMENTATION FOR PAYMENT (NOV 1998)

G.2 ADMINISTRATIVE CONTRACTING OFFICE

G.4 TECHNICAL DIRECTIONS/RELATIONSHIP WITH USAID

G.5 PAYING OFFICE

G.6 ACCOUNTING DATA AND APPROPRIATION DATA

SECTION H - SPECIAL CONTRACT REQUIREMENTS

H.1 NOTICE LISTING CONTRACT CLAUSES INCORPORATED BY REFERENCE

H.2 AIDAR 752.7004 EMERGENCY LOCATOR INFORMATION (JULY 1997)

H.3 FOREIGN GOVERNMENT DELEGATIONS TO INTERNATIONAL CONFERENCES (JAN 2002)

H.4 INSURANCE AND SERVICES

H.5 AIDAR 752.228-70 MEDICAL EVACUATION (MEDEVAC) SERVICES (JUL 2007)

H.7 AUTHORIZED GEOGRAPHIC CODE

H.8 LOGISTIC SUPPORT

H.9 CONSENT TO SUBCONTRACT

H.10 ADS 302.3.5.5 REPORTING OF FOREIGN TAXES (JULY 2007)

H.11 ADS 302.3.5.14(a) USAID DISABILITY POLICY - ACQUISITION (DEC 2004)

H.12 AIDAR 752.225-70 SOURCE AND NATIONALITY REQUIREMENTS (FEB 2012)

H.13 AIDAR 752.245-71 TITLE TO AND CARE OF PROPERTY (APR 1984)

H.14 AIDAR 752.7007 PERSONNEL COMPENSATION (JULY 2007)

H.15 INFORMATION TECHNOLOGY REQUIREMENT

H.16 MANAGEMENT OF INFORMATION TECHNOLOGY RESOURCES

H.17 VALUE ADDED TAX AND CUSTOMS DUTIES

H.18 AIDAR 752.7032 INTERNATIONAL TRAVEL APPROVAL AND NOTIFICATION REQUIREMENTS (APR 2014)

H.19 LANGUAGE REQUIREMENTS

H.20 EMPLOYMENT COSTS OF THIRD COUNTRY NATIONALS AND COOPERATING COUNTRY NATIONALS . 39

H.21 ENVIRONMENTAL COMPLIANCE AND MANAGEMENT

H.22 ADS 302.3.5.9 NONDISCRIMINATION (JUNE 2012)

H.23 DISCLOSURE OF INFORMATION

H.24 ELECTRONIC PAYMENTS SYSTEM

H.25 SUPPORT TO THE GOVERNMENT OF UGANDA

H.26 NON-DISCRIMINATION IN IMPLEMENTATION OF USAID-FUNDED PROGRAMS

H.27 BUSINESS CLASS TRAVEL

H.28 USAID IMPLEMENTING PARTNER NOTICES (IPN) PORTAL FOR ACQUISITION

(JULY 2014)

H.29 USAID-FINANCED THIRD-PARTY WEB SITES (AUGUST 2013)

H.31 PROHIBITION ON CONTRACTING WITH ENTITIES THAT REQUIRE CERTAIN INTERNAL CONFIDENTIALITY

AGREEMENTS-REPRESENTATION (APR 2015)

H.32 SUBMISSION OF DATASETS TO THE DEVELOPMENT DATA LIBRARY (DDL)

PART II - CONTRACT CLAUSES

SECTON I – CONTRACT CLAUSES

I.1 52.225-25 - PROHIBITION ON CONTRACTING WITH ENTITIES ENGAGING IN CERTAIN ACTIVITIES OR

TRANSACTIONS RELATING TO IRAN-REPRESENTATION AND CERTIFICATIONS (DEC 2012)

I.2 52.233-2 SERVICE OF PROTEST (SEP 2006)

I.3 752.202-1 DEFINITIONS (JAN 1990)

I.5 752.211-70 LANGUAGE AND MEASUREMENT (JUN 1992)

I.6 752.245-70 GOVERNMENT PROPERTY - USAID REPORTING REQUIREMENTS

(JUL 1997)

I.7 752.7006 NOTICES (APR 1984)

I.8 752.7008 USE OF GOVERNMENT FACILITIES OR PERSONNEL (APR 1984)

I.9 752.7025 APPROVALS (APR 1984)

1.10 752.209-71 ORGANIZATIONAL CONFLICTS OF INTEREST DISCOVERED AFTER AWARD (JUN 1993)

1.11 752.231-71 SALARY SUPPLEMENTS FOR HOST GOVERNMENT EMPLOYEES

(MAR 2015)

1.12 752.7034 ACKNOWLEDGMENT AND DISCLAIMER (DEC 1991)

1.13 752.228-7 INSURANCE - LIABILITY TO THIRD PERSONS (JUL 1997)

1.14 752.229-70 FEDERAL, STATE AND LOCAL TAXES

1.15 752.7010 CONVERSION OF U.S. DOLLARS TO LOCAL CURRENCY (APR 1984)

1.16 752.7013 CONTRACTOR-MISSION RELATIONSHIPS (OCT 1989)

1.17 752.7027 PERSONNEL (DEC 1990)

1.18 752.7029 POST PRIVILEGES. (JUL 1993)

1.19 752.7033 PHYSICAL FITNESS (JUL 1997)

1.20 752.7004 EMERGENCY LOCATOR INFORMATION (JUL 1997)

1.21 752.7032 INTERNATIONAL TRAVEL APPROVAL AND NOTIFICATION

REQUIREMENTS (APR 2014)

1.22 752.7015 USE OF POUCH FACILITIES (JUL 1997)

(g) When the contract includes a requirement for the contractor to assign copyright to the government or another party, the contracting officer shall incorporate (48 CFR) 752.227-14 and/or include an express limitation or restriction in the contract.

USAID contracting officers will assert such a right in limited circumstances in accordance with the principles as stated in

(48 CFR) 27

SECTON J – SECTION J - LIST OF DOCUMENTS, EXHIBITS AND OTHER ATTACHMENTS

SECTION K - REPRESENTATIONS, CERTIFICATIONS, AND OTHER STATEMENTS OF

BIDDERS

K.1 NOTICE LISTING SOLICITATION PROVISIONS INCORPORATED BY REFERENCE

K.2 52.204-8 ANNUAL REPRESENTATIONS AND CERTIFICATIONS (MAY 2014)

K.3 INSURANCE - IMMUNITY FROM TORT LIABILITY

K.4 52.204-3 TAXPAYER IDENTIFICATION (OCT 1998)

K.5 FAR 52.209-5 CERTIFICATION REGARDING DEBARMENT, SUSPENSION, PROPOSED DEBARMENT, AND

OTHER RESPONSIBILITY MATTERS (APRIL 2010)

K.6 FAR 52.209-7 INFORMATION REGARDING RESPONSIBILITY MATTERS (JUL 2013)

K.7 52.230-1 COST ACCOUNTING STANDARDS NOTICES AND CERTIFICATION (MAY 2012)

K.8 52.230-7 PROPOSAL DISCLOSURE--COST ACCOUNTING PRACTICE CHANGES (APR 2005)

K.9 COMPLIANCE WITH VETERANS EMPLOYMENT REPORTING REQUIREMENTS

K.11 SIGNATURE

SECTION L – INSTRUCTIONS, CONDITIONS AND NOTICE TO OFFERORS

L.1 TYPE OF CONTRACT

L.2 PROPOSAL INSTRUCTIONS

L.3 GENERAL INSTRUCTIONS TO OFFERORS

L.4 INSTRUCTIONS FOR THE PREPARATION OF THE TECHNICAL PROPOSAL

L.5 PRICE PROPOSALS

M.1 GENERAL INFORMATION

M.2 TECHNICAL PROPOSAL EVALUATION CRITERIA

M.3 INSTITUTIONAL CAPACITY [SECTION L.4.2.4]

M.4 PRICE PROPOSAL EVALUATION

PART 1 – SCHEDULE

SECTION B - SUPPLIES OR SERVICES AND PRICE/COSTS

B.1 PURPOSE

The purpose of this performance and impact evaluation is to evaluate the extent to which Literacy Achievement and Retention Activity (LARA) (Cooperative Agreement AID-617-A-15-00009)’s intended outcomes are being achieved and the effectiveness of program approaches and interventions. The impact evaluation will test the extent to which the program’s activities improve reading skills and retention in primary grades.

B.2 CONTRACT TYPE

This is a Firm-Fixed Price (FFP) completion type contract. For the consideration set forth below, the Contractor will provide the deliverables or outputs described in Section C and F, in accordance with performance standards specified in Section E.

B.3 TOTAL PRICE AND OBLIGATED AMOUNT

(a) The total fixed price for the performance of the work required hereunder is [TBD- amount to be inserted upon award].

(b) The contractor shall not exceed the aforesaid obligated amount.

(c) Funds obligated here under are anticipated to be sufficient through TBD.

[END OF SECTION B]

SECTION C – STATEMENT OF WORK

C.1 OVERVIEW

C.1.1 Activity Title:

“Performance and Impact Evaluation for the Literacy Achievement and Retention Program” (LARA-P&IE)

C.1.2 Contract Purpose

The United States Agency for International Development Mission in Uganda (USAID/Uganda), in collaboration with the Africa Bureau, Office of Sustainable Development, Education Division (AFR/SD/ED) requires technical assistance to conceptualize and conduct a performance and impact evaluation of the five-year Literacy Achievement and Retention Activity (LARA) (Cooperative Agreement AID-617-A-15-00009, awarded April 1, 2015). The purpose of this performance and impact evaluation is to evaluate the extent to which intended outcomes are being achieved and the effectiveness of program approaches and interventions. The impact evaluation will test the extent to which the program’s activities improve reading skills and retention in primary grades. A particular focus will be the activity’s impact on student safety and school-related gender-based violence (SRGBV) and to what extent these result in measurable impact on student learning, specifically early grade reading.1 The results are expected to be used by USAID, other development organizations, and local stakeholders to inform policy and program design in the education sector. Additionally, the evaluation will provide evidence necessary for effective dialogue with policy makers and other education stakeholders to raise awareness of the seriousness of SRGBV.

Within 30 days of the award, the LARA-P&IE Contractor will begin working closely with USAID and the LARA Implementing Partner (IP) to identify the best way to evaluate retention in recognition of concerns about the availability and reliability of data and other constraints. The USAID Contracting Officer’s Representative (COR) will approve the design option that is selected to evaluate retention as well as the overall design of the P&IE. The LARA P&IE Contractor will identify, collect and analyze data to support the impact evaluation of Results 1 and 2. (EGRA data will be collected by the LARA IP). A final data collection plan will be approved by the COR as part of the overall P&IE design and work plan. It is crucial that USAID, the LARA IP, and the LARA P&IE Contractor work closely together to create a viable evaluation design.

C.2 BACKGROUND

C.2.1 School Related Gender-Based Violence2 One factor contributing to poor student retention is unsafe learning environments. In developing countries, school-related gender-based violence (SRGBV) is especially prevalent. In a survey conducted in Uganda, more than 33 percent of school children reported having been in a physical fight during the school year.3 Other research reports that children may be subjected to a range of school violence including corporal punishment, humiliation, fighting, sexual harassment and assault, and bullying based on gender, sexuality, disability, stigmatized illness, or minority group status. Direct and indirect experiences of school violence can also lead to school avoidance, low student participation in class, an inability to concentrate in class, and poor student attitudes towards school. Though sparse, some studies have reported an association between the safety of schools and educational achievement. The patchy evidence that school violence has a negative impact on educational achievement is compelling and a relationship that USAID seeks to further explore.

The USAID Africa Bureau is addressing SRGBV through the Opportunities for Achievement and Safety in Schools (OASIS) program, which aims to conduct and disseminate research further investigating a causal link between non-violent, safe schools and learning through a series of analytic and outreach activities. OASIS will conduct rigorous evaluations of safe schools intervention programs in a select number of USAID sub-Saharan countries to test their impact on educational achievement. The LARA P&IE represents a major effort to support the goals of OASIS.

C.2.2 USAID Evaluation a Policy and Required Data Reporting

1 USAID acknowledges that measuring for a causal relationship may be too complex for the scope of this contract given the linkages assumed in the LARA Results Framework between SRGBV and retention, and retention and educational achievement as measured through early grade reading assessment outcomes. Evaluation questions will be developed in consultation with the LARA IP and USAID. The Contractor is responsible, in consultation with the LARA IP, and USAID, for determining which relationships are appropriate and realistic to measure.

2 SRGBV includes physical, sexual or psychological violence or abuse that is based on gendered stereotypes or that targets students on the basis of their sex, sexuality or gender identities. The underlying intent of this violence is to reinforce gender roles and perpetuate gender inequalities. It includes rape, unwanted sexual touching, unwanted sexual comments, corporal punishment, bullying, and verbal harassment. Unequal power relations between adults and children and males and females contribute to this violence, which can take place in the school, on school grounds, going to and from school, or in school dormitories and may be perpetrated by teachers, other school staff, students, or community members. Both girls and boys can be victims, as well as perpetrators. SRGBV results in sexual, physical, or psychological harm to girls and boys.

3 World Health Organization.2003.Global school-based student health survey Uganda.

USAID introduced an Evaluation Policy in 2011 emphasizing that to be effective stewards of public resources, USAID’s policy and investment decisions must be based on the best empirical evidence available. USAID’s evaluation policy aims to increase our focus on rigorous data collection and evaluation techniques.

The USAID Policy defines performance evaluations as those evaluations that focus on descriptive and normative questions:

what a particular project or program has achieved (either at an intermediate point in execution or at the conclusion of an implementation period); how it is being implemented; how it is perceived and valued; whether expected results are occurring; and other questions that are pertinent to program design, management and operational decision making.

Performance evaluations often incorporate before-after comparisons, but generally lack a rigorously defined counterfactual.

According to USAID policy an impact evaluation measures the change in a development outcome that is attributable to a defined intervention: impact evaluations are based on models of cause and effect and require a credible and rigorously defined counterfactual to control for factors other than the intervention that might account for the observed change. Impact evaluations in which comparisons are made between beneficiaries, who are randomly assigned to either a treatment or a control group, provide the strongest evidence of a relationship between the intervention under study and the outcome measured.

The USAID Evaluation Policy specifies that “In cases where impact evaluations are undertaken to examine the relationship between an intervention or a set of interventions and changes in a key development outcome, a parallel contractual or grant agreement will be established at the inception to accompany implementation.” It is with this intention that USAID/Uganda has released this RFP for the LARA P&IE.

C.2.3 USAID/Uganda Mission and Africa Bureau Collaboration on Ideas and Learning

A central principle to USAID/Uganda’s programming approach is the Collaborating, Learning and Adapting Agenda (CLA).

The CLA Agenda acknowledges that program design and implementation occur in dynamic contexts, that information is often incomplete and that policy and implementation environments change. The CLA Agenda aims to ensure that as programs are designed, a deliberate attempt is made for systematic information gathering, reflection, and adaptation of USAID programs to reflect Uganda’s evolving context and needs. The Africa Bureau’s Office of Sustainable Development is also charged with generating new ideas that respond to global development challenges and is collaborating with USAID/Uganda on the evaluation contract through its OASIS program. The evaluation activities envisioned under this impact evaluation contract support the Mission’s CLA Agenda and the Africa Bureau’s OASIS program with the goal to advance research specifically related to SRGBV and reading outcomes. Because SRGBV is such a severe problem, with research sorely needed, exceptional collaboration, learning, and ongoing adaptations by the LARA Implementing Partner (IP), the LARA-P&IE Contractor, and USAID are essential.

C.2.4 Literacy Achievement and Retention Activity

LARA’s purpose is to improve key aspects of reading skills in both English and three local languages in the early primary grades (Result 1); and improve the retention of learners in the primary grades (Result 2). The LARA program description serves as the primary technical reference for the LARA P&IE. LARA and the LARA P&IE will collaborate closely and directly with each other, and this collaboration will extend to the Ministry of Education, Science, Technology, and Sports (MoESTS) and USAID to develop the evaluation design, P&IE data collection systems, and the selection procedures required for an experimental or quasi-experimental impact evaluation.

Because LARA is results based, the P&IE Contractor will not have full knowledge of planned interventions and sample size until after the P&IE award is issued. Upon award of the two instruments, the two partners, together with USAID and MoESTS, must work collaboratively to create and agree upon a rigorous evaluation design for LARA Results 1 and 2, including the definition of carefully selected indicators that provide meaningful measures of performance and impact that support both the overall evaluation goals of USAID’s international policies and guidelines, and USAID/Uganda in particular.

C.2.5 LARA Results Framework

Results Framework

LARA will focus on achieving the Results and Intermediate Results in the Results Framework. The finalized Activity Results Framework must lend itself to impact evaluation through a clear articulation, to the extent possible, of the causal linkages building from sub-IRs to Program Results.

One major focus of the impact evaluation will be to test the hypothesis that if SRGBV is reduced, then children’s learning (as measured through the early grade reading assessment (EGRA) tools) will improve. The LARA P&IE Contractor will identify, collect, and analyze data to support the impact evaluation of Results 1 and 2. (EGRA data will be collected by the LARA IP).

A final data collection plan will be approved by the COR as part of the overall P&IE design and work plan. Below is a brief summary of LARA Results 1 and 2.

Result 1: Improved reading skills in three local languages and English for primary grade learners.

LARA will scale-up and implement the Early Grade Reading model (EGR) in 28 districts. This will include training teachers, tutors, and district officials, providing support supervision for teachers, as well as printing and distributing teacher and pupil reading materials4 in 3 local languages and English. Special emphasis will be placed on collecting data and information on Primary 4, the transition year from local language to English. The LARA IP must collect and analyze information on key factors contributing to a successful transition from the use of the local language to English as the main means of gaining academic information and skills. Increased reading comprehension will be the focus of P4 interventions. The activity will continue to strengthen MoESTS support-supervision processes to ensure teachers receive sufficient support as they use the materials and put into practice the newly acquired teaching methodologies in the classroom. Additionally, the activity will develop and carry out community and household level interventions to promote literacy in communities and at homes by mobilizing youth and family members.

Indicators:

The LARA P&IE Contractor will provide technical assistance to the LARA IP and USAID in revising the Results Framework as needed, in selecting impact indicators, and in developing the Activity Monitoring and Learning Plan.

Required Data Collection Activities:

4 Instructional materials are being designed by the School Health and Reading Program.

Activity Objective: Increased reading skills for 1.3 million students.

Result 1: Improved reading skills in 3 local languages and English for early primary grade learners.

Result 2: Improved retention in primary grades.

IR 2.1: Reduced number of incidents of SRGBV IR 2.2: Increased number of schools with Safe Classroom environments

IR 1.1: Improved capacity of MoESTS and education system to effectively sustain the national roll-out of the Early grade reading model IR 1.2: Improved reading skills for learners in P1-P4.

IR 1.3: Improved supervisory support and mentorship for primary teachers.

IR 1.4: Increased community and household level involvement in promoting literacy attainment.

Sub-IR 2.1.1 Improved prevention, response, and reporting capacity of schools, MoESTS, and education system Sub-IR 2.1.2 Increased school, community, youth, and parental involvement in SRGBV recognition, prevention, response, and monitoring Sub-IR 2.2.1 Increased school, community and parental involvement in promoting retention and safety in and around schools Sub IR 2.2.2 Improved capacity of education system to sustain retention interventions

The full lists of activities for LARA are in the program description. The LARA P&IE Contractor will identify, collect, and analyze data to support the impact evaluation of Result 1, incorporating available information and data from MoESTS and other sources. EGRA data will be collected by the LARA IP and these data will be accessible to the P&IE Contractor.

Result 2: Improved retention in primary grades.

Promoting a safer primary learning environment is expected to increase retention because learners will be able to focus on their lessons and feel secure enough to stay in school. Ensuring a safe learning environment includes non-violent classroom management, positive discipline, safeguarding of children’s rights, and the prevention, response and reporting of school-related gender-based violence. 5

The activity will focus on introducing classroom management practices that create a safer learning environment.

Community-based approaches will be developed or strengthened to promote accountability for students’ safety and to increase retention rates. These interventions must be fully coordinated with other USAID IPs working in the child protection sector. Training will be provided for teachers, parents, community members, and learners on how to curb gender-based violence and other forms of abuse within the school system.

Indicators:

Impact indicators will be selected in collaboration with USAID and the LARA IP.

Required Data Collection Activities:

The LARA IP will conduct a rapid assessment to identify the most important causes of primary students “dropping out,” focusing on critical vulnerabilities for children at schools---special attention will be given to vulnerabilities faced by girls and SRGBV. The P&IE Contractor will have the responsibility for identifying and collecting data for the Result 2 impact evaluation, incorporating information and data from MoESTS and other sources. Overall primary survival rate improvements will not be available during the life of the activity because the seven-year primary cycle in Uganda is longer than the length of the activity.

Expected Outcomes:

An overall decrease in learners’ exposure to violence, including physical, sexual, and psychosocial, in schools and nearby communities will lead to improved retention for learners, providing overall support to literacy interventions.

C.2.6. Activity Monitoring and Evaluation for LARA

Project monitoring and evaluation (M&E) for LARA is a requirement of the LARA IP, who will track a range of indicators (input, output, and outcome) that are relevant for each of its components and consistent with USAID’s performance monitoring and evaluation policies, including financial data to permit costing and financial analysis. The P&IE Contractor will contribute to identifying essential data necessary to support activity monitoring and evaluation. The LARA IP will be responsible for conducting Early Grade Reading Assessments (EGRAs) and sharing those data with the LARA P&IE Contractor. Best practices will be documented and presented to appropriate forums. The LARA IP is expected to work closely and continuously with the P&IE Contractor to coordinate, plan, and implement a literacy and retention learning agenda as well as the performance and impact evaluation. The LARA P&IE will also conduct Data Quality Assessments for all LARA indicators.

C.3. EVALUATION RATIONALE

Primary reading achievement levels in Uganda are dismal, with alarming implications for long-term economic growth, health outcomes, and democratic governance. At this point in the development of Uganda’s early grade reading approach, solid impact data are crucial to support the efforts of the MoESTS and the three MoESTS-backed early grade reading activities that USAID is supporting in 85 of Uganda’s 112 districts. The need for a focus on Result 2 is based on current knowledge of school safety and school-related gender-based violence (SRGBV), which has been associated with lower enrollment, low and intermittent attendance, and higher drop-out rates. A recent literature review suggests that SRGBV also negatively impacts learning outcomes, suggesting that SRGBV may be a significant obstacle in the way of achieving USAID’s Education Strategy Goals 1 and 3, improved reading skills for 100 million children in primary grades (Goal 1) and increased equitable access to education in crisis and conflict environments for 15 million learners (Goal 3). However, the relationship

5 USAID’s Doorways materials are a necessary reference:

http://www.usaid.gov/documents/1865/doorways-training-manual-school-related-gender-based-violence-prevention-and-response http://www.usaid.gov/documents/1865/doorways-training-manual-school-related-gender-based-violence-prevention-and-response between SRGBV and educational outcomes is still not well understood. The evidence that does exist comes largely from correlational studies or smaller-scale studies from developed countries, and is largely restricted to bullying but does not include rigorous studies of other types of SRGBV. One of the specific recommendations of the aforementioned literature review is to conduct a number of impact evaluations of programs designed to reduce SRGBV in developing countries. Such impact evaluations would enable a better understanding of the relationship between SRGBV and learning.

The purpose of the impact evaluation is to test the extent to which the program’s Result 1 activities improve primary reading skills, and the extent to which Result 2 activities intended to promote student safety and reduce SRGBV improve student learning, specifically early grade reading. The overall development hypothesis for this activity is if USAID/Uganda supports early-grade reading and complementary retention interventions, including reduction of SRGBV, and works through MoESTS systems at all levels to strengthen their capacity to sustain EGR methodology implementation at scale, then better quality educational services will become more accessible, higher literacy levels will be achieved, and primary retention rates will increase, providing foundational support to broad-based economic growth, health outcomes, and democratic governance.

A key element of the development hypothesis is that reduced incidence of SRGBV leads to improved retention and improved learning as measured by early grade reading test scores. The evaluation will enable a better understanding of the mechanisms and impact of SRGBV, especially on learning. Understanding the link between SRGBV and learning is critical for future programming: if SRGBV prevents students from learning, then interventions focused on academic issues, by themselves, will not be sufficient.

Impact evaluation questions for Results 1 and 2 will be developed and finalized by the P&IE Contractor, the LARA IP, and USAID during the early collaboration phase.

C.4 EVALUATION DESIGN AND METHODOLOGY

The P&IE Contractor will execute a descriptive and analytical performance evaluation employing qualitative and quantitative methods of data collection. Qualitative methods will include document review; key informant interviews at the national, district, and school level, focus group discussions, and school and classroom observation. Monitoring data and performance feedback from all activities will be considered. Quantitative data collected through EGRA and other sources will be developed to meet the need of the impact evaluation. Purposeful sampling of schools and districts will be employed. Data quality and analysis will be validated through triangulation of multiple sources and stakeholders. Performance evaluation questions covering management, learning, design, implementation, results, sustainability, and the effective use of funds will be developed in consultation with the LARA IP and USAID.

The P&IE Contractor’s impact evaluation design will address the extent to which quantitative and qualitative data can be used, and how to measure the impact of the five-year LARA activity on student learning, retention, and safer classrooms (i.e., reduced school-related gender-based violence); it will, where possible, include comparative analysis of the factors contributing to program results. It is anticipated that impact will be measured at the level of the program’s target schools, districts, and at the national level. There will be both treatment and control schools. The estimated total number of schools in the 28 target districts is 3,847; this number will be clarified as part of the rapid start-up assessment done by LARA. All treatment schools will receive both Results 1 and 2 activities, and the method of selecting control schools will be determined by USAID and the LARA IP in consultations with the P&IE Contractor and MoESTS, considering the lessons learned from the School Health and Reading Program. The design must propose the method and timing to integrate control schools into the activity (i.e. receive treatments).

The regulations and processes of the Government of Uganda review boards that regulate research at all levels will be clearly identified and followed by the P&IE Contractor, and included in the Work Plan.

The evaluation design shall clearly define how impact will be defined and measured and examine the strength of the development hypotheses and causal relations that formed the basis for the Activity Results Framework. Because the impact evaluation is intended to advance a global research agenda, it shall be designed to address the information and management needs of the MoESTS and the USAID/Uganda Mission, but also to further USAID’s Agency-wide priorities for literacy and the prevention and response to school-related gender-based violence, within implementation constraints.

It is expected that an experimental or quasi-experimental design will be the primary methodology used, depending on selection of program and control schools, which must be coordinated with the LARA IP. Data on reading skills are expected to be made available by the LARA IP. The contractor is expected to develop methodologies and tools to measure SRGBV, retention, and other appropriate aspects of both Results 1 and 2, and collect supporting impact data in coordination with the

LARA IP and USAID.

Additionally, impact evaluations conducted in isolation from other sources of information are vulnerable both technically and in terms of their potential effectiveness. Therefore, data collection and evaluation will include contextual factors, such as changes in policy, other interventions in the same districts, and others. While impact evaluations may produce reliable estimates of the causal effects of the safe school package of interventions, they aren’t designed to provide insights into the program implementation. Because of this, the performance evaluation and specific use of qualitative data will be required to examine aspects of the safe school interventions, in order to inform and interpret the results from the impact evaluation.

Complementary qualitative research will produce more comprehensive answers to the evaluation questions.

The evaluation design and corresponding methodologies will be developed jointly with the LARA Program IP, USAID and MoESTS after the award of the P&IE contract.

C.5 EVALUATION PRODUCTS

In collaboration with the USAID COR and LARA, the Contractor is expected to develop the methodology and analysis required to execute the contract, and specifically the following deliverables:

Contract Deliverables Year 1 Evaluation Design and Overall Activity Work Plan Year 1 2 Data Collection Tools Report Year 1 3 Baseline Enumerator Training and Reporting

3.A Training Plan 3.B Training Report

Year 1

4 Baseline Impact and Performance Data Collection Year 1 5 Baseline Impact and Performance Data Report Year 2 6 Mid-term Impact and Performance Data Collection Year 2-3 7 Mid-term Impact and Performance Evaluation Year 3 8 Endline Enumerator Training and Reporting

7.A Training Plan 7.B Training Report

Year 4

9 Endline Impact and Performance Data Collection Year 5 10 Endline Impact and Performance Data Report Year 5 11 Final Impact and Performance Evaluation Year 5

Learning Workshops for Evaluation Findings with a CLA focus (see above C.2.3, and below C.5.11):

12.A. Mid-term Learning Workshops 12.B. Final Learning Workshops 12.C. Policy Brief document

Year 3 Year 5 Year 5

C.5.1 Evaluation Design and Overall Activity Work Plan (Deliverable 1)

USAID’s Evaluation Policy recognizes that the ability to conduct performance and impact evaluations requires that key elements of project evaluation design must be coordinated from the program’s inception. USAID’s purpose in issuing two parallel instruments—one for program implementation and the other for evaluation—is to enable consultation and collaboration in ensuring that the critical elements for evaluation design are jointly considered and agreed upon by all stakeholders from the outset of the program. It is expected that upon award of these instruments, the two implementers shall communicate and work together, consulting USAID and MoESTS, to develop a rigorous evaluation design and plan.

The collaborative process, its outcomes and challenges must be documented in a detailed design and work-plan document to be approved by the USAID COR. The evaluation design and plan will, at a minimum, address the following components:

A. Design Rationale The LARA program description will guide the P&IE Contractor regarding the design rationale of the LARA P&IE, and the P&IE Contractor shall work with LARA and USAID to define key areas to be addressed in terms that shall enable measurement, performance and impact evaluation. The design rationale shall describe the research/evaluation activities that are explicit and/or implicit to the activity design, and the opportunities and constraints of the implementing context of the activity.

B. Finalized Program Results Framework

Changes to the Activity Results Framework may be proposed by the P&IE Contractor to strengthen the foundation for rigorous performance and impact evaluations. It shall clearly differentiate between the Activity Results, the Intermediate Results that are necessary to accomplish those results, and proposed activities that will lead to Intermediate and Activity Results. The results framework will also clearly identify the most critical assumptions and other changes in the operating environment that may influence the activity outcomes. It is important that this Activity Results Framework clearly illustrate possible causal relationships to justify the selection of the impact evaluation designs, recognizing that inter-dependency dynamics between sub I.R.s, feedback loops and participant attrition rates may also require special considerations

C. Selection of Subjects and Comparisons

The selection of subjects largely determines potential impact evaluation designs and methodologies, including the distinction between experimental and quasi-experimental design. The selection of subjects must necessarily be guided by the program’s planned strategic approaches and treatments, which shall also influence the determination of how to constitute a comparison or control group. The selection of subjects will be done by the LARA IP in consultations with the P&IE Contractor, USAID and MoESTS. Both the locus and mode of selecting subjects shall be clarified, including whether to use centralized locus and randomized selection, or instead to choose a quasi-experimental impact evaluation design. This section shall discuss the selection of either a comparison or control group, or explain in detail any other evaluation methodology agreed upon for assessing the counterfactual. If using a quasi-experimental design, this section shall discuss threats to internal/external validity of the proposed design, and propose means of countering those threats when selecting subjects and comparisons.

D. Description of Treatment(s)

In partnership with the LARA IP and the MoESTS, the research design/evaluation plan shall briefly summarize the treatment(s) planned under Results 1 and 2. If multiple treatments are intended for the same group, a description shall be included for each of the key treatments that shall be assessed as separate factors contributing to program impact. To the extent that different treatments are intended for different groups (i.e., within target schools or districts vs. at the national level), differences between those treatments shall be clearly articulated. Treatment(s) shall be described concisely, but with adequate detail on component activities to serve as the basis for formulating questions and methodologies for the performance evaluation, and to draw useful conclusions regarding effective program design based upon the findings of the impact evaluation. The design must propose the method and timing to integrate control schools into the activity (i.e. receive treatments).

E. Data Collection, Quality, and Measurement Systems

The P&IE Contractor will describe and define the impact data collection and measurement systems that will be used at a level of detail that is adequate to ensure the capacity for rigorous impact evaluation. This requires consultation among all stakeholders including MoESTS on which measurement systems shall be designed, using which data collection tools, with what frequency, and from which subjects and comparison groups. The impact measurement systems must generate baseline and endline data adequate to inform the impact evaluation. Decisions on data collection will also consider USAID reporting requirements. The LARA P&IE Contractor will identify, collect, and analyze data to conduct the impact evaluation of Results 1 and 2. (EGRA data will be collected by the LARA IP).The P&IE Contractor, USAID, and the LARA IP will jointly determine data collection for the impact evaluation and how to triangulate data collected by LARA. Any additional data collection planned by the P&IE Contractor shall be described in this section.

Baseline, midline and endline data quality assessments (DQAs) will be done for LARA by the P&IE Contractor in consultation with USAID and the LARA IP and reported in the quarterly and annual reports.

Data on SRGBV shall include data on attitudes and opinions relating to SRGBV as well as gender-based violence and gender equality more broadly, frequency and type of corporal punishment, bullying, physical violence, sexual harassment and sexual violence. Data collection on SRGBV needs to be coordinated with data collection on learning outcomes to ensure that both data collections are coordinated between LARA and the LARA P&IE, if possible, and executed by the same enumerators for the purpose of efficiency. Data collection on school enrollment, attendance, retention and completion and learning outcomes will include EGRA assessments in line with the Education Strategy Implementation Guidance and Technical Notes. All data shall be disaggregated by sex; other factors by which to disaggregate (such as geographical location, age, or urban vs. rural areas) may be determined. Baseline data collection will be done before program interventions begin and endline data during year five; all data collection timelines will be coordinated among stakeholders, with timelines adjusted to meet implementation constraints such as school terms and MoESTS standardized testing.

The approach and effectiveness of LARA regarding USAID’s District Operational Plans and youth issues will be documented and evaluated by the P&IE Contractor.

The Evaluation Design and Activity Work Plan will define roles and responsibilities of different parties, and include an implementation timelime of the five-year life of the program.

Deliverable 1 must be technically approved by USAID and officially submitted to the USAID Contracting Officer’s Representative (COR) within three months of program start-up. It is expected that the initial rapid assessment done by LARA will yield important information for the design of the P&IE evaluation and may necessitate some adjustments to the Work Plan.

C.5.2 Data Collection Tools Report (Deliverable 2)

The P&IE Contractor must define the types of assessment tools, instruments and processes needed to yield quantitative and qualitative data (where appropriate and needed), and select or develop appropriate assessment tools for obtaining information about safe schools, including identification of the most common factors that lead to SRGBV, is coordination with the School Health and Reading Program and the LARA IP.

Assessment tools are required for the baseline and endline data collections. Where possible and appropriate, these tools will be identified from existing USAID assessment tools, in consultation with the LARA IP6. Other assessment tools available through the international community that do not require copyright royalty payments or other user fees may also be considered. If the P&IE Contractor identifies key areas that cannot be assessed through existing tools, the development of new assessment tools may be necessary, which will require pre-testing and possible revisions. The P&IE Contractor will identify and assess the relevance of assessment tools to those factors generally considered related to safe schools. The Contractor will recommend additional assessment tools, survey-type instruments, key informant type instruments, or other methodological approaches when appropriate and determine whether or not they need to be developed.

Deliverable 2 must be technically approved by USAID and officially submitted to the USAID Contracting Officer’s Representative within four months of program start-up.

C.5.3 Baseline Enumerator Training and Reporting (Deliverable 3)

3.A. The P&IE Contractor shall develop a plan for recruiting and training of enumerators in a timely fashion, so that training is completed prior to data collection. Methods for assessing enumerators’ ability to administer the surveys in a child- and gender-sensitive manner shall be included. Using LARA and other previously-trained enumerators will be considered.

3.B. A report on the actual training shall be submitted.

Deliverable 3.A will be submitted within four months of program start-up. 3.B will be submitted to the USAID Contract Officer’s Representative for approval within one month after completion of training.

C.5.4 Baseline Impact and Performance Data Collection (Deliverable 4)

Prior to the beginning of the baseline data collection, the Contractor will coordinate with the LARA IP to identify effective referral and after-care services that are in place to support students who may be identified by participating in the baseline.

Baseline impact data will be collected before the start of treatments. The LARA P&IE Contractor will identify and collect baseline data to support the performance and impact evaluation of Results 1 and 2 in close coordination with USAID and the LARA IP. (EGRA data will be collected by the LARA IP).

6 USAID does not maintain a list of its assessment tools. Internet searches of IPs’ website may yield a range of assessment tools developed under a contract with USAID.

Deliverable 4, the baseline impact and performance data collection, will be completed before the start of treatments and will be described in the work plan and evaluation design, and reported as described below in C.5.5 as part of Deliverable 5.

C.5.5 Baseline Impact and Performance Data Report (Deliverable 5)

The content format and presentation of the baseline report will be determined in consultation with the USAID COR.

Deliverable 5 must be submitted to the USAID Contracting Officer’s Representative for approval within three months after the baseline data collection.

C.5.6 Mid-Term Impact and Performance Data Collection (Deliverable 6)

The LARA P&IE Contractor will identify and collect baseline data to support the performance and impact evaluation of Results 1 and 2 in close coordination with USAID and the LARA IP. (EGRA data will be collected by the LARA IP).

Deliverable 6, the baseline impact and performance data collection, will be described in the work plan and evaluation design, and reported as described in C.5.7 as part of deliverable 7. The data collection will be completed one month before the midline date.

C.5.7 Mid-Term Impact and Performance Evaluation Report (Deliverable 7)

The P&IE Contractor will execute a mid-term performance evaluation employing qualitative and quantitative methods of data collection as described above in C.4. Deliverable 7 will include a mid-term data collection section reporting on Deliverable 6.

Deliverable 7 must submitted to the USAID COR three months after the midline date.

C.5.8 End line Enumerator Training and Reporting (Deliverable 8)

8.A. The P&IE Contractor shall develop a plan for recruiting and training of enumerators in a timely fashion, so that training is completed prior to data collection. Methods for assessing enumerators’ ability to administer the surveys in a child- and gender-sensitive manner shall be included. Using LARA and other previously-trained enumerators will be considered.

8.B. A report on the actual training shall be submitted.

Deliverable 8.A will be submitted six months prior to the enumerating training. 8.B will be submitted to the USAID Contract Officer’s Representative for approval within one month after completion of training.

C.5.9 Endline Impact and Performance Data Collection (Deliverable 9)

Endline impact and performance data will be collected within the first six months of Year 5.

For Deliverable 9 the endline impact data collection will be described in the work plan and evaluation design, and reported as described below.

C.5.10 End line Impact and Performance Data Report (Deliverable 10)

The content format and presentation of the end line findings will be based on what is approved for the previous reports.

This detailed report shall contain an executive summary, analysis of impact evaluation findings and implications for USAID and the MoESTS, and any written and/or visual presentations…