Attachment J.5 EGRP Final Report.pdf

Government of Nepal Ministry of Education, Science, and Technology

October 31 2020

This publication was prepared for review by the United States Agency for International Development.

It was prepared by RTI International.

Assistance to Basic Education— All Children Reading (ABE-ACR) USAID’s Early Grade Reading Program (EGRP) in Nepal Final Progress Report, Years 1–5: March 2015–October 2020

PUBLIC REPORT

Assistance to Basic Education— All Children Reading (ABE-ACR) USAID’s Early Grade Reading Program (EGRP) in Nepal

Final Progress Report, Years 1–5: March 2015–October 2020

PUBLIC REPORT

Contract No.: AID-367-TO-15-00002

Prepared for:

Shannon Taylor Director, Education Development Office United States Agency for International Development/Nepal GPO Box No. 295 U.S. Embassy, Maharajgunj Kathmandu, Nepal E-mail: sjtaylor@usaid.gov

Prepared by:

RTI International 3040 E. Cornwallis Road Post Office Box 12194 Research Triangle Park, NC 27709-2194 www.rti.org

The authors’ views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

Cover photo: Early grade students going home after a productive day at school, Kanchanpur. Photo credit: Merit Maharjan. All other photos in the report are by Merit Maharjan unless otherwise noted.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – II –

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – III –

Table of Contents

Executive Summary

Program Overview

Program Results Monitoring of Outputs

Major Activities and Achievements Result 1: Improved Early Grade Reading Instruction

Sub-Result 1.1 Evidence-based early grade reading instructional materials designed, distributed, and in use Sub-Result 1.2 In-service professional development for teachers in public schools on reading instruction and the use of the materials provided Sub-Result 1.3 Monitoring and coaching for teachers in early grade reading instruction provided Sub-Result 1.4 Classroom-based and district-based early grade reading assessment processes improved

Result 2: Improved National and District Early Grade Reading Service Delivery Sub-Result 2.1 Early grade reading data collection and analysis systems improved Sub-Result 2.2 Policies, standards, and benchmarks that support improved early grade reading instruction institutionalized 22 Sub-Result 2.3 Planning and management of financial, material, and human resources devoted to early grade reading improved Sub-Result 2.4 National standards for early grade reading improvement adopted and geographically expanded

SUCCESS STORY

Result 3: Increased Family and Community Support for Early Grade Reading Sub-Result 3.1 Community awareness of the importance of early grade reading and language for reading instruction as appropriate increased Sub-Result 3.2 Family engagement to support reading increased Sub-Result 3.3 PTA/SMC ability to contribute to quality reading instruction increased Sub-Result 3.4 Parent and community capacity to monitor reading progress increased

SUCCESS STORY

Monitoring and Evaluation Sub-Result 4.1 Develop and implement an internal M&E system for regular program monitoring Sub-Result 4.2 Conduct EGRA/EMES Assessment Sub-Result 4.3 Update PMP and Capture Data for PMP Indicators Sub-Result 4.4 M&E Capacity Building of EGRP Staff

Systems Strengthening and Sustainability Gender Equity and Social Inclusion

Lessons Learned and Recommendations

Conclusions

List of Acronyms and Abbreviations

List of Figures

Figure 1: EGRP by the numbers Figure 2: Nepal at a glance Figure 3: EGRP intervention districts Figure 4: EGRP results framework Figure 5: Summary of key endline findings Figure 6: Endline results: Average oral reading fluency, non-Nepali versus Nepali speakers, Cohorts 1 and 2 combined, by grade Figure 7: EGRP research themes Figure 8: Achievement of PMP targets Figure 9: Result 1 activity flow Figure 10: Initial training and orientation by the numbers Figure 11: Categories of participants in the LEU capacity-development workshops (N = 6,290) Figure 12: Online dashboard for data from school monitoring visits Figure 13: Sample infographic for EGRP midline evaluation data, 2018, for USAID Early Grade Reading Barometer Figure 14: Post-decentralization education structures supported by EGRP Figure 15: NEGRP stakeholder profile Figure 16: EGRP GESI approach Figure 17: Participation in social and community mobilization activities, by caste/ethnicity

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – IV –

Figure I: EGRP by the numbers

1 1,985

8,099

14,31 1

551,439

1 1,728,075

3,744 schools administrators (22% women) teachers (56% women) learners (53% girls) educational materials parent-teacher associations (PTAs)/ school management committees (SMCs)

Grants to I 5 partner NGOs and 1,500 SMCs

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 1 –

EGRP supported Nepal’s journey to self-reliance by:

Promoting open and accountable governance through transparent information systems and citizen participation in decision making around early grade reading.

Building government capacity to expand the country’s own NEGRP at all levels, from the local to the national.

Promoting inclusive development by creating teaching and learning approaches and materials that were tailored for diverse language groups and by ensuring wide participation in parent and community engagement activities.

Strengthening civil society capacity by closely collaborating with local partner nongovernmental organizations (NGOs).

Building citizen capacity by improving education quality through improved reading outcomes in the early primary grades.

This final report highlights the progress and accomplishments of the United States Agency for International Development’s (USAID’s) Early Grade Reading Program (EGRP) in Nepal.

Supported with generous funding from the American people through USAID, EGRP was a program of technical assistance to the Government of Nepal (GON).

The program was implemented by RTI International and its subcontractors, Another Option, Plan Nepal, Room to Read, and SIL LEAD. EGRP’s goals were to improve the reading skills of public primary school students in grades 1–3 and to demonstrate an early grade reading model that could be scaled up throughout the country, in support of Nepal’s own National Early Grade Reading Program

(NEGRP).

Implemented from March 2, 2015, to October 31, 2020, EGRP achieved these goals through three main focus areas: improved early grade reading instruction;

strengthened national, district, and local-level early grade reading service delivery;

and increased family and community support for early grade reading.

Figure 1: EGRP by the numbers

Executive Summary

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 2 –

Implemented in 16 districts covering all of Nepal’s 7 provinces, EGRP rolled out in two cohorts: Cohort 1, which covered schools in 6 districts, and Cohort 2, which included schools in 10 districts (see map on page 6 for more details).

Cohort 1 schools received more intensive support between 2016 and 2018, while interventions in Cohort 2 schools ramped up between 2018 and 2020.

EGRP’s achievements related to improved early grade reading instruction included supporting the GON’s Curriculum Development Center (CDC) to create an evidence-based early grade reading model; a grades 1–3 curriculum based on that framework; associated teaching and learning materials (TLMs), especially teacher’s guides and pupil books; and supplementary reading materials (SRMs) aimed at various reading levels and in several languages. All four components would enhance teachers’ instructional approaches and children’s reading practice. To promote gender equality and social inclusion (GESI – see text box below), EGRP helped to advance materials development and teacher training for mother-tongue instruction and to devise teaching and learning approaches for children who speak Nepali as a second language (L2). The program also introduced guidance on gender sensitivity in TLM development. In addition, because it was clear that teacher training in the country could use a boost, the program worked closely with the Center for Education and Human Resource Development (CEHRD)1 to develop and roll out a teacher professional development (TPD) package as well as a teacher professional support (TPS) system to operationalize the new curriculum and materials.

EGRP achieved a number of goals related to strengthened national, provincial, district, and local-level early grade reading service delivery, especially in terms of improving systems for collecting and analyzing data on early grade reading. The program collaborated with CEHRD to incorporate information about spoken languages into Nepal’s Integrated Education Management Information System (IEMIS). They also developed a profile tool that Local Education Units (LEUs) could use to compile basic educational information linked with the IEMIS, and later use the data for decision making. EGRP engaged extensively on policy issues with the Ministry of Education, Science and Technology (MOEST)—called the Ministry of Education (MOE) until 20182—and its constituent agencies. Together, they improved standards and benchmarks for early grade teachers and learners and developed a costed “minimum package” of early grade reading interventions that the GON could feasibly roll out nationally through the NEGRP.

EGRP was implemented against a backdrop of substantial changes in the Nepali context, including a new Constitution in 2015 that established a novel system of decentralized governance. To support decentralization, EGRP led NEGRP orientation and capacity development across newly decentralized Ministries of Social Development (MOSDs) and other structures at the provincial level;

Nepal’s decentralized governance levels

Established by a new Constitution in 2015, these levels include:

6,743 wards palikas* districts provinces

Languages and GESI in EGRP

Nepal is a multiethnic nation with 123 different mother tongues spoken, and Nepali as the official language. Learning assessments have shown that children who do not speak Nepali as a first language tend to struggle with developing early grade reading skills more than their Nepali-speaking peers. Nepal also ranked 101 out of 153 nations on the 2020 Global Gender Gap Index. EGRP implemented GESI strategies related to curriculum and materials development and parent and community engagement, among others, to help address these disparities.

The NEGRP Minimum Package

The elements recommended in the minimum package are:

1 CEHRD was established in 2018, combining the former Department of Education (DOE), the Non-Formal Education Center (NFEC), and the National Center for Educational Development (NCED), which was renamed the Human Resource Development Division (HRDD).

2 For ease of reading, this report uses “MOEST” throughout, even for activities that preceded the Ministry’s name change in early 2018.

Curriculum

TLMs

Teacher training

Community mobilization

Student assessment

Monitoring and evaluation

(each with detailed costing).

*Palikas are local governments—also referred to as urban or rural municipalities.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 3 –

Education Development Coordination Units (EDCUs)3 at the district level; and LEUs as well as other government structures at the local level. In the final program year—July 2019–October 2020—local governments were inspired to allocate over US$200,000 of their own resources toward early grade reading activities, such as creating print-rich environments in schools and individual classrooms and conducting reading festivals.

Achievements related to increased family and community support for early grade reading included extensive social and behavior change (SBC) efforts, such as campaigns through radio programs, text messages, posters, a video documentary, and mobile application platforms. EGRP supported local structures, such as District Core Teams (DCTs), mayors and deputy mayors, and ward chairs—as well as “Tole” (community) Reading Groups (TRGs) and early grade reading subcommittees formed under the school management committees (SMCs), with more than 100,000 total members. The point was to better understand, plan for, and implement core community and parent engagement activities to promote early grade reading. In addition, EGRP worked closely with local partner nongovernmental organizations (NGOs) to implement community mobilization activities, such as peer education and early grade reading-related events, for the parents and guardians of children in grades 1–3. These efforts reached 181,935 participants in total. They were bolstered by grants that EGRP awarded to the SMCs, which schools used to improve the quality of their learning environments in pursuit of strengthened early grade reading.

All of these achievements were underpinned by a robust monitoring and evaluation (M&E) system. The system ensured frequent quality monitoring as well as data gathering to assess whether the program was achieving the targets it was supposed to, and whether it was having the desired impacts on the intended beneficiaries.

EGRP’s M&E-related work used cutting-edge technology to depict data visually in a compelling format for audiences at all levels—from the local to the international.

The M&E team also ensured that the national-level Education Review Office (ERO) would have the capacity to conduct assessments, such as the classroom-based early grade reading assessment (CB-EGRA), an instrument and process that engaged teachers, parents, and GON stakeholders at the local and central levels in gathering and using real-time information about children’s learning outcomes.

EGRP implemented activities in close coordination with USAID’s separate funding commitment directly to the Nepali government (i.e., a government-to-government [G2G] investment) for early grade reading, as well as other inputs jointly financed by the GON and international development agencies. Together with EGRP, these investments supported development and implementation of a 2016–2023 School Sector Development Plan (SSDP) that would serve as a multiyear road map to education improvements.

3 EDCUs were formerly called District Education Offices For ease of reading, this report uses EDCU throughout.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 4 –

EGRP helped the GON and USAID generate substantial evidence about the effectiveness of the NEGRP model as well as areas for future improvement. The impact evaluation demonstrated that the program had positive effects at the endline among learners in Cohorts 1 and 2, as related in the final impact evaluation report, prepared by NORC at the University of Chicago. The findings for Cohort 2 at endline were similar to the effects found at midline for Cohort 1, where the program had already been fully rolled out. These gains were comparable to those achieved in other similar USAID-supported reading programs around the world.

In addition to NORC’s report, analyses that RTI undertook showed that although Cohort 2 saw a significant increase, the impact on reading skills in Cohort 1 did not change significantly from midline to endline. This finding suggests a plateau effect, in which Cohort 1 achieved substantial improvements by midline but would have required additional ongoing support to help teachers continue improving their instructional practices and increase children’s learning even further.

EGRP’s impact evaluation also demonstrated that native Nepali speakers (first-language or L1 learners) gained more than their nonnative Nepali-speaking (L2) peers, indicating that L2 students would need targeted support to close the gap with their L1 peers.

In addition, the program’s impact evaluation identified promising effects on instructional practices indicators that EGRP helped develop for Nepal, such as the teachers’ reading instruction index; teachers’ early grade reading knowledge, skills, and attitudes index; and schools’ remediation practices for struggling early grade readers. Moreover, comparing the baseline and endline assessment results, the impact evaluation highlighted improvements in parents’ beliefs and attitudes regarding the importance of early grade reading for their children, as well as increases in positive practices to promote reading at home.

Challenges and lessons learned pinpointed the need for practical strategies related to language of instruction in classrooms, and particularly a coherent, evidence-based approach to mother tongue and Nepali L2 issues. In addition, EGRP’s research and monitoring efforts pointed to the need to ensure that early grade reading best practices are well-supported as a new integrated curriculum is rolled out. They also underscored the importance of focusing explicitly on teacher training and professional support in promoting key early grade reading-related instructional changes.

In its final years, EGRP generated substantial learning around challenges and potential local solutions for TPS, an area that suffered some disarray during the move to decentralization. The lessons learned should be carried forward in future education sector policies and investments if—as is assumed—Nepal intends to achieve sustained improvements in early grade reading teaching and learning.

In light of the COVID-19 pandemic, but also more broadly, EGRP identified significant demand for increased, equitable use of information technology— as feasible in the Nepal context—to promote improved early grade reading instruction and school-community linkages.

Finally, EGRP’s experience pointed to the critical importance not just of instructional and systems change, but also of widespread efforts to communicate in ways that could change society’s behavior in support of children’s reading.

EGRP impacts

With United States Government (USG) assistance, the percentage of learners who demonstrated reading fluency and comprehension of grade-level text at the end of grade 2 increased from 0.9 percentage points at baseline in 2016, to 3.14 percentage points at midline in 2018, to 3.21 percentage points at endline in 2020.

In addition, the average number of correct words per minute (cwpm) read by second graders increased from 6.9 at baseline to 9.96 at midline and 12.15 at endline, a near doubling in 4 years. This represents progress toward Nepal’s national early grade oral reading fluency benchmark of 45 cwpm.

Elements of the Integrated Curriculum

Key elements of the new integrated curriculum are:

Retains subject distinctions, such as Nepali, mathematics, and English

Introduces themes that cut across subjects, such as “me and my family” and “our community”

Incorporates “soft skills” such as interpersonal, communication, and citizenship skills throughout

NEPAL Landlocked South Asian nation bordered by China and India

Population of 28 million in 2018 126 castes/ethnicities 123 different mother tongues

Ranked 147th out of 189 nations on the Human Development Index 101 out of 153 nations on the Global Gender Gap Index

I 18.8% primary gross enrollment rate 7% primary repetition rate

73.5% cohort persistence rate to grade 5

55% of grade 2 children whose mother tongue was not Nepali scored zero on an oral reading fluency assessment in 2014.

2015 Constitution led to sweeping decentralization with persistent governance, capacity, and human resources gaps at lower levels

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 5 –

Program Overview

Figure 2: Nepal at a glance

EGRP was a 5-year program of technical assistance to the GON with two overarching goals: (1) to improve the reading skills of public primary school students in grades 1–3 in 16 target districts; and (2) to support the GON through Phase 1 of its own NEGRP, completing the design and demonstration of a national model that the GON could then implement nationwide within its budget.

Based in Kathmandu and working in the target 16 districts (see Figure 3), EGRP supported MOEST and its Central Level Agencies (CLAs)—the CDC, CEHRD,4 and ERO—to develop an early grade reading program that the GON could replicate and expand to all districts of the country in a cost-effective and sustainable manner.

Key Central Level Agencies

CDC – Curriculum Development Center. Mainly responsible for curriculum and materials development.

CEHRD – Center for Education and Human Resource Development.

Mainly responsible for annual strategic planning and implementation, as well as for teacher professional development and support, through its Human Resource Development Division

(HRDD).

ERO – Education Research Office. Mainly responsible for research and assessment.

4 The GON organized the CEHRD after EGRP began implementation, as part of the shift to federalism. As noted in the Executive Summary, it combines the former DOE, the NCED (now HRDD), and the NFEC.

EGRP GOAL

Public primary students in the early grades (1-3) in the 16 target districts with improved reading skills

Result I: Improved Early Grade Reading Instruction

Sub-Result 1.1: Evidence-based early grade reading instructional materials designed, distributed, and in use.

Sub-Result 1.2: In-service professional development for teachers in public schools on reading instruction and the use of these materials provided.

Sub-Result 1.3: Monitoring and coaching for teachers in early grade reading instruction provided.

Sub-Result 1.4: Classroom-based and district-based early grade reading assessment processes improved.

Result 3: Increased Family and Community Support for Early Grade Reading

Sub-Result 3.1: Community awareness of the importance of early grade reading and language for reading instruction as appropriate increased.

Sub-Result 3.2: Family engagement to support reading increased.

Sub-Result 3.3: PTA/SMC ability to contribute to quality reading instruction increased.

Sub-Result 3.4: Parent and community ability to contribute to quality reading instruction increased.

Result 2: Improved National and District Early Grade Reading Service Delivery

Sub-Result 2.1: Early grade reading data collection and analysis systems improved.

Sub-Result 2.2: Policies, standards, and benchmarks that support improved early grade reading instruction institutionalized

Sub-Result 2.3: Planning and management of financial, material, and human resources devoted to early grade reading improved.

Sub-Result 2.4: National standards for early grade reading improvement adopted and geographically expanded.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 6 –

Figure 3: EGRP intervention districts

EGRP’s goals were achieved through meeting a set of results, as shown in Figure 4.

Figure 4: EGRP results framework

6 EGRP Cohort 1 Districts

10 EGRP Cohort 2 Districts

Province Boundary

EGRP Central Office

EGRP Regional Office

Rukum West

Rukum East

Baglung

Nawalparasi East

Nawalparasi West

Rupandehi

Kanchanpur

Mahot-tari

Solukhumbu

Okhaldhunga

Sankhuwasabha

Udayapur

Dhading

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 7 –

Key EGRP activities centered on (1) collaborating with the GON to develop and distribute TLMs and SRMs; (2) developing teacher training manuals and delivering training; and (3) bolstering the system of teacher professional support in the early grades. In addition, the program provided technical assistance to strengthen the MOEST’s IEMIS and built capacity of District Administration Offices, EDCUs, and LEUs to effectively manage school-based early grade reading activities. EGRP also sensitized and mobilized families and communities to support early grade reading activities in schools and communities.

Finally, EGRP assisted the GON to design and conduct reading assessments in the primary grades, plus other ongoing formative classroom assessments and national-level assessments. Their purpose was to improve measurement and decision making regarding children’s learning outcomes in the first three grades of primary school.

EGRP’s technical approach to implementing the program emphasized:

• Providing embedded and other direct technical assistance services to the GON to implement and continue the evolution of the GON’s NEGRP Plan. This included supporting the GON to harmonize and organize early grade reading approaches that were already being implemented around the country.

• Maintaining EGRP’s consistency with GON NEGRP standards, making it feasible for the GON to replicate and implement EGRP innovations on a national scale. This involvement included both applying and helping to improve the policies, systems, and resources that were available to the GON.

• Employing multiple first-language instructional methods and materials according to dominant L1, in selected languages in addition to Nepali.

• Employing evidence-based strategies to increase family and community support for early grade reading.

EGRP was led by RTI International and had four subcontractors over its period of performance, each of which played a significant role in implementing the program, as follows:

• RTI was responsible for overall leadership and management of EGRP, ensuring timely, high-quality implementation of program activities, and overseeing staffing and financial resources; ·

• Another Option’s contributions focused on early grade reading social and behavioral change communication and advocacy (primarily Result 3);

• Plan Nepal oversaw much of EGRP’s community sensitization and mobilization work (Result 3);

• Room to Read provided expertise in early grade reading policy, materials, and curriculum development, as well as teacher training (Results 1 and 2); and

• SIL LEAD applied its expertise in mother-tongue development to EGRP’s curriculum and instruction activities (especially Sub-Result 1.1).

Program Results

EGRP’s outcomes were measured through an external impact evaluation that drew upon the student reading performance data that the program collected at baseline in 2016, at midline in 2018, and at endline in 2020 in schools and with students in grades 1–3.

Baseline data were collected between February and May 2016 (with an interruption for end-of-year exams), midline data were collected between February and March 2018, and endline data between February and March 2020. A maximum of 285 schools and 7,994 children were included in the evaluation sample (exact numbers varied at each phase). Data collection instruments included the EGRA tool plus student background questions that had been adapted to the Nepali context for the 2014 national-level EGRA, and an Education Management Efficiency Survey (EMES).5

The baseline evaluation identified low skill levels overall in areas such as letter and matra6 identification, and very low performance in nonword decoding, oral reading fluency, and both reading and listening comprehension. L2 learners scored substantially below L1 learners on all EGRA subtasks.

The midline evaluation found that EGRP had positive effects on all skills for L1 and L2 learners in Cohort 1 schools, with the largest effects for L1 students, and particularly those in grades 1 and 3. The midline identified that the program tended to benefit L1 learners more than L2 learners and had not closed the large gap between the two groups that was identified via the baseline data. In addition, the midline showed positive program impacts on the availability of student textbooks in EGRP-supported classrooms;

and improvements in the reading instructional practice index in Cohort 1, but no effects in Cohort 2.

Finally, the midline uncovered that EGRP had led to an improvement of almost one point out of 14 in the management index for Cohort 1, but again no impacts in Cohort 2.

In addition to NORC’s reporting on endline findings, RTI conducted extensive analyses of the endline data to identify actionable recommendations to inform future interventions as EGRP was coming to a close.

As indicated in the summary in Figure 5, the endline findings were similar to those revealed at midline for Cohort 1, and showed significant improvement from midline in Cohort 2.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 8 –

5 Sitabkhan, Y., & DeStefano, J. (2014). Nepal Early Grade Reading Assessment (EGRA) study. Prepared for USAID under Education Data for Decision Making (EdData II), Data for Education Programming in Asia and Middle East, Task Order No. AID-OAA-BC-11-00001 (RTI Task 15). Research Triangle Park, NC: RTI International. https://pdf.usaid.gov/pdf_docs/PBAAF535.pdf

6 The Nepali writing system includes consonants as well as independent and dependent vowels. A matra is a combination of a consonant plus a dependent vowel sign that together form a syllable.

The EGR.P interventions for both Cohort I and Cohort 2 had a significant impact on student reading skills at endline, with gains comparable to those achieved in other USAID-support reading programs.

The impact on Cohort I reading skills at endline remained unchanged from midline.

The impact on Cohort 2 reading skills at endline was significantly higher than at midline.

Native Nepali speakers (LI learners) gained more than their nonnative Nepali-speaking (L2) peers.

There was a significant reduction in the percentage of nonreaders; however, there was only a small percentage change in the number of students reading fluently with comprehension.

Impact on learning outcomes is explained by the results from approximately one-third of the schools.

Figure 5: Summary of key endline findings

The positive impact overall in both cohorts indicated that the EGRP model could improve reading outcomes despite several contextual factors that hampered implementation. For example, with regard to the plateau effect witnessed in Cohort 1, EGRA/EMES data, together with M&E data and field research, revealed that teachers had taken up some, but not all, of the instructional practices on which they were trained, and that even this degree of uptake was uneven among teachers. Such findings suggest that fidelity of implementation of instructional practice had not improved sufficiently among the Cohort 1 schools to continue to increase learning outcomes.

One of the most crucial elements to strengthen instruction is for knowledgeable advisors to provide ongoing support to teachers—another area raised at midline as needing improvement. After the midline, CEHRD began implementing a new TPS model. Subsequently, however, one of the key actors in that model, who was mandated to provide technical assistance and advice to other TPS actors— called a Resource Person—was removed from the system. As a result, a continued lag in fidelity is not highly surprising.

Looking specifically at the findings in relation to Nepali L1 and L2 students, the endline results were similar to the midline results: L1 students performed significantly better than L2 students, and L1 students had greater gains from baseline to endline (see Figure 6). These findings indicate that, in effect, L1 students benefited more from the EGRP interventions than L2 students.

It is worth noting that after seeing results at midline that revealed the gap for L2 students, EGRP worked with CDC to develop materials that would provide guidance and strategies for teachers to use for supporting L2 learners. The same guidance and strategies were also integrated into refresher training for Cohort 2. Unfortunately, the materials and revised training were completed only by the beginning of 2020, which means that they were not yet incorporated into EGRP implementation before the endline. As such, the L1/L2 gap at endline was similar to that seen at midline and was not unexpected.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 9 –

Grade 1

L2 Speakers

LI Speakers

L2 Speakers

LI Speakers

Grade 2

Grade 3

L2 Speakers

LI Speakers

0 10 20

By the end of grade 2, non-Nepali speakers were on average a grade behind their

Nepali-speaking peers

Figure 6: Endline results: Average oral reading fluency, non-Nepali versus Nepali speakers, Cohorts 1 and 2 combined, by grade

These findings also demonstrate that the explicit and systematic instruction promoted by EGRP benefited students at the lower end of the distribution. This finding is typical of what has been noted in other new, large-scale reading programs, where lower level skills improve first and more quickly.

Finally, in addition to the findings related to reading outcomes, RTI’s preliminary analysis of endline data highlighted that teachers had continued to improve their early grade reading instructional practices, knowledge, skills, and attitudes. The impact evaluation also uncovered that there was a plateau effect in schools’ remediation practices to provide supplemental support for struggling students. At the same time, there was largely positive movement on a school leadership and management index for early grade reading between baseline and endline. In addition, in a testament to the strength of EGRP’s parent and community engagement approach, a large percentage of parents or guardians reported reading to their children at least once a week, a figure that climbed steadily from baseline to endline.

EGRP Performance Evaluation

USAID commissioned an external performance evaluation of EGRP, conducted by CAMRIS through USAID Nepal’s Monitoring, Evaluation, and Learning activity, with a baseline in 2016 and a follow-up in 2018. This evaluation examined the extent to which EGRP had demonstrated a viable and sustainable early grade reading model and strengthened GON service delivery to implement the model.

The follow-up report identified that EGRP had demonstrated viable components of an early grade reading model. However, elements such as motivated teachers, support for non-Nepali-speaking children, and orientation and training on NEGRP implementation at local levels were identified as critical for sustained success. The researchers also found that no EGRP Results or Sub-Results had achieved “institutionalized” status by 2018, but that several elements had become “initially established”: evidence-based instructional materials; in-service professional development; data collection and analysis systems and use of data for decision making; policies, standards, and benchmarks; planning and management; and family engagement to support reading.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 10 –

Attitudes toward Time-tabling for Feasibility of cell Early grade Teacher and student Teacher motivation Sustainability of language of instruction early grade reading phone applications reading public language mapping community engagement for early grade reading for early grade reading awareness

Evaluation of Effectiveness of the

Rana Tharu mother teacher professional tongue pilot support system in early grade reading

The impact and performance evaluations were supplemented by an array of research studies conducted by EGRP from 2015 through 2020, on key topics related to all three Results, as summarized in Figure 7.

Figure 7: EGRP research themes

Monitoring of Outputs The foundation of EGRP’s monitoring system was a Performance Monitoring Plan (PMP) with 19 indicators. Figure 8 captures EGRP’s achievement against the program’s indicator targets.

Figure 8: Achievement of PMP targets

No. Indicator Achievement

1 Percent of learners who demonstrate reading fluency and comprehension of grade-level text at the end of grade 2 with U.S. Government (USG) assistance 3.21%

2 Number of learners reached in reading programs at the primary level with USG assistance 551,439

3 Number of primary school educators who complete professional development activities on implementing evidence-based reading instruction with USG assistance 14,311

4 Number of primary or secondary textbooks and other teaching and learning materials (TLMs) provided with USG assistance 11,728,075

5 Number of education administrators and government officials who complete professional development activities with USG assistance 8,185

6 Number of PTAs or community governance structures engaged in primary or secondary education supported with USG assistance 3,744

Numbers of policies / regulations / administrative procedures in each of the following stages of development as a result of USG assistance in each case: 1. Analysis, 2. Stakeholder consultation / public debate, 3. Drafting or revision, 4. Approval (legislative or regulatory), 5. Full and effective implementation

8 Number of civil society organizations receiving USG assistance engaged in advocacy interventions 15

9 Percentage of grade 2 teachers that scored 9 or more out of 13 score on a reading instructional index, demonstrating fidelity to effective early grade reading teaching practices 80.1%

10 Percentage of schools that score 5 or above out of 7 on the remediation practice index, demonstrating supplemental support for struggling students 78.5%

11 Percentage of schools that deliver reading instruction in a language spoken and understood by the majority of students (including Nepali as a second language) 52.5%

12 Percentage of parents or guardians who report reading to their children or listening to their children read to them at least once a week 96.2%

13 Average grade 2 oral reading fluency (cwpm) scores among students in USAID/Nepal’s EGRP target districts for Nepali language 12.2

14 Percentage of target schools allocating at least 1 hour a day to reading instruction (grades 1–3) 49.0% 15 Percentage of local organizations with improved capacity and/or performance scores 100%

16 Percentage of schools that score at least 10 of 14 total possible points on the school leadership and management index, demonstrating active support for early grade reading 72.3%

17 Number of schools reporting progress toward standards for reading performance 4,426

18 Percentage of parents and school management committee members expressing positive attitude toward early grade reading and mother tongue 90.2%

19 Percentage of teachers who score 18 or above out of 25 on a knowledge / skills / attitudes index 35.5%

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 11 –

Major Activities and Achievements

This section describes EGRP’s major activities and achievements from 2015 to 2020. The section is structured to follow along with EGRP’s results framework, including the three Results and each of their Sub-Results, as captured in EGRP’s goals (achieved through meeting the Results shown in Figure 4 above). Activities and achievements in M&E, sometimes referred to as EGRP’s fourth Result, are also discussed.

Result 1: Improved Early Grade Reading Instruction Result 1 was at the core of EGRP’s programmatic effort to improve early grade reading instruction. Under this Result, key EGRP activities focused on:

• developing, printing, and distributing early grade TLMs and SRMs;

• developing teacher training manuals and delivering training on the use of materials;

• providing orientation and training for head teachers to support their teachers in enhancing their skills; and

• conducting EGRAs and other ongoing classroom assessments.

This section describes activities and achievements under Result 1’s four Sub-Results, related to early grade reading instructional materials, TPD, teacher monitoring and coaching, and assessment.

Sub-Result 1.1 Evidence-based early grade reading instructional materials designed, distributed, and in use Figure 9 gives an overview of how activities under Result 1 flowed.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 12 –

Key Achievements:

Sub-Result 1.1

Developed and revised Nepali L1 TLMs for grades 1–3

Distributed 2,523,897 TLMs to schools

Distributed 9,470,106 SRMs to schools

Developed grades 1–3 TLMs for Rana Tharu, Awadhi, and Maithili mother tongues

Distributed and piloted Rana Tharu grades 1 and 2 TLMs in schools

Distributed Awadhi and Maithili grade 1 TLMs to EDCUs

Provided technical assistance for grades 1–3 integrated curriculum development, including TLMs and curriculum guides; offered orientations to the new curriculum

Developed and distributed teacher resource materials to support Nepali L2 children

Supported CDC to develop a new Nepali type font and materials library

How IRI Works Instructional Materials Design, Printing, and Distribution

Technical Assistance to CDC

Support with Design

Printing Support

Packaging and Dispatch

Delivery to School

Figure 9: Result 1 activity flow

Nepali L1 Curriculum and TLMs

EGRP’s support to CDC in revising the Nepali L1 curriculum and TLMs for grades 1–3 began in 2015. The process included initial orientations, revision of expected grades 1–3 learning outcomes, review of existing materials developed by other organizations in Nepal, selection of the international organization Room to Read’s materials for grades 1–2, and revision of those materials by early 2016. Grade 3 materials were not developed at the time, as Room to Read’s resources covered only the first two grades.

The grades 1–2 materials, along with SRMs for grade 3, were printed and distributed to all schools in Cohort 1 districts for the 2016/2017 school year. In November of 2015, EGRP also distributed SRMs for around 5,800 schools in 14 districts of Nepal that had been affected by earthquakes earlier in the year. EGRP then supported CDC to develop grade 3 TLMs, revise some of the TLMs for grades 1–2, and distribute those materials to schools for the 2017/2018 school year.

CDC requested EGRP assistance to again revise the TLMs for grades 1–3 and develop additional TLMs such as decodable books, big books, letter and window cards, word and picture cards, pinwheel cards, and consonant–vowel and rhyme charts. These were finalized and distributed to all schools in the Cohort 1 and Cohort 2 districts for the 2018/2019 school year.

After this, with EGRP support, CDC completed another round of revision of the TLMs based on feedback from monitoring and operations research, and CDC developed new SRMs. The Nepali TLMs and the new titles of SRMs were printed and distributed to all schools in 16 districts by April 2019 for use in the 2019/2020 school year.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 13 –

Support for Integrated Curriculum Materials

In addition to EGRP’s technical assistance with materials development, in 2019 and 2020, the program aided CDC in developing a new integrated curriculum for grades 1–3. This assistance included illustration and design work on the grades 1–3 TLMs and feedback and revisions to the grade 1 materials after they had been piloted. EGRP also provided technical capacity-building workshops on developing materials for Nepali L2 learners and theme-based materials development to align with the cross-cutting thematic approach of the new curriculum.

In 2020, EGRP supported CDC to conduct provincial-level orientations on the integrated curriculum to ensure widespread understanding of how to begin implementing it in the 2020/2021 school year and beyond.

Supplementary Reading Materials

To help promote a culture of reading and improve teachers’ instructional practices around reading in the early grades, EGRP supported CDC to develop, print, and distribute SRMs starting in 2016. This effort included reviewing existing SRMs for aspects such as gender and inclusion, selecting 149 titles, and leveling those books so that 66 titles would be identified for libraries in each grade (including some books that covered multiple grades). EGRP then assisted with some editing, retyping, design, and illustration where needed, and developed a book register and instructions to teachers on how to use the SRMs.

In 2018, EGRP supported CDC to edit and finalize an additional 28 SRMs that they had developed, for a total of 177 early grade materials. The final copies were printed and distributed to all schools in the 16 EGRP districts, along with TLMs, for use in the 2019/2020 school year. In 2019, CDC requested EGRP to revise all of the titles to reflect new Nepali spelling rules, and reprint and distribute SRMs to all schools in the 16 districts; both tasks were completed late that year. In 2020, USAID requested EGRP to begin distributing the SRMs to 22 new NEGRP districts. By the end of the program in October 2020, 6,968 schools in these districts had received SRMs, and the remaining schools in those districts would receive them under a new program.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 14 –

Types of teaching and learning materials provided

Student Workbooks

Teacher’s Guides

Decodable Books

Big Books

Pinwheel Charts

Letter Flash Cards

Window Cards

Picture Word Cards

Wooden Letter Cubes

Rhymes Chart

Consonant–Vowel Chart

Mother-Tongue TLMs and SRMs

Starting in its first year, along with Nepali language materials, EGRP focused on supporting CDC to develop a new set of TLMs for grades 1, 2, and 3 for selected mother tongues. This process took longer than expected due to concerns about the practicalities of rolling out a mother-tongue approach, and a national-level decision to shift from a bilingual instructional approach to use of mother tongues as a subject.

The process of developing mother-tongue TLMs and SRMs included training on SynPhony software for linguistic analysis (letter and word frequency), and scope and sequence development for Rana Tharu grades 1–3. Rana Tharu TLMs for grade 1 were piloted in the 2018/2019 school year in Kailali and Kanchanpur districts. The grades 2 and 3 Rana Tharu scope and sequence were then developed and the grade 1 scope and sequence revised, and TLMs for grade 2 were created and rolled out during the 2019/2020 school year.

Starting in 2018, EGRP provided additional training in SynPhony and further capacity development to writers of mother-tongue materials. After this, writers developed TLMs for Awadhi and Maithili grades 1–3 from 2019 to 2020, and they continued creating materials for Rana Tharu grade 3 and revising the Rana Tharu grade 1 TLMs.

At the same time, CDC availed EGRP’s assistance with layout and design of 40 titles of Nepali SRMs that had been translated into each mother tongue.

Development of Teacher Resource Materials to Support Nepali L2 Readers

In early 2019, CDC agreed to work on developing Nepali L2 teacher resource materials. To support this undertaking, EGRP conducted training for CDC and provided expertise on Nepali L2 teaching and learning approaches.

A draft of the teacher resource materials was developed in early 2020 and finalized based on feedback by the end of the program, and appropriate teaching and learning approaches for L2 learners were also incorporated into refresher trainings for Cohort 2 teachers held in the same year. The aim of the materials was to help children overcome the range of barriers they face if they speak Nepali as a second language, including aspects such as pronunciation, grammar, reading fluency, and reading comprehension.

Other Activities

EGRP also supported CDC to develop a new Nepali type font that would enable CDC to maintain uniformity in the font used for early grade reading materials. This work was completed in late 2019.

In addition, EGRP helped strengthen GON libraries for early grade reading by providing resource materials in hard and soft copies to CDC; HRDD; and provincial, district, and local education offices.

What is a scope and sequence?

Based on research and evidence about the way children learn, a scope and sequence is an integral part of any curriculum. It defines the scope, or range, of knowledge and skills to be covered. It also lays out the sequence, or the order and progression, of knowledge and skills to be addressed.

EGRP in Nepal, Final Progress Report, March 2015–October 2020 – 15 –

3,109 teachers (1,283 men and 1,826 women) received the initial 7-day in-service training in 2016 on the use of Nepali TLMs for grades I and 2.

EGR.P also supported a 2-day orientation for head teachers and SMC members on early grade reading principles so they could effectively support their teachers.

4,952 head teachers (3,686 men and 1,266 women)

3,177 SMC members (2,255 men and 922 women) in all 6 Cohort I districts received an orientation on the importance of early grade reading.

Sub-Result 1.2 In-service professional development for teachers in public schools on reading instruction and the use of the materials provided

7-Day Teacher Training and Orientation for Head Teachers and SMC Members

In addition to the curriculum-related activities under Sub-Result 1.2, EGRP supported the GON to improve in-service professional development for early grade reading. Training included orientation for the then-NCED7 on evidence-based practices in early grade reading instruction, as well as planning for development of training packages. EGRP supported NCED to develop and roll out master training of trainers (MTOTs) from Cohort 1 districts in early 2016, with trainings implemented subsequently at district level, and then for clusters of schools grouped together to ensure that a manageable number of trainees participated in each.

Figure 10: Initial training and orientation by the numbers

3+5-Day Teacher Professional Development “Bridge Course”

A 3-day library management and follow-up training—which came to be called the “Bridge Course”—was planned for the teachers who had received initial training on the use of the new materials in 2015, to allow them to complete NCED’s 10-day professional development credit (7 previous days of training plus 3 additional days).

Subsequently, however, a change in NCED leadership shifted the plans for teacher training, and it was agreed that the TPD module should consist of 10 days of face-to-face and 5 days of school-based training, meaning that previously trained teachers would need to receive 3 additional face-to-face training days plus 5 days of school-based training. A total of 1,651 teachers in Cohort 1 districts received the Bridge Course training in Year 3 of the program.

10+5-Day Teacher Professional Development Course

Starting from the second year of the program (2016–2017), EGRP supported NCED to develop and roll out the 10+5-day TPD approach for teachers who had not previously received the 7-day training and Bridge Course.