Attachment_J.17_Section_C_Education_Data_Activity.pdf

Attachment J.17 – Section C USAID Education Data Activity

SECTION C – DESCRIPTION/SPECIFICATIONS/PERFORMANCE BASED

WORK STATEMENT

C.1 ACTIVITY TITLE

The activity title is USAID EDUCATION DATA.

C.2 PURPOSE

The purpose of the USAID Education Data Activity is to provide distinct assessment, data management, research and evaluation services to monitor and track the progress of USAID’s

Let’s Read Zambia bilateral activity (formerly known as Sustainable and Effective Education

Delivery).

C.3 BACKGROUND

Zambia‘s education system faces daunting challenges in its ability to provide adequate and quality education services. A high pupil-teacher ratio (59:1), high pupil-textbook ratio (10 pupils per book), and extremely poor learning outcomes in critical areas such as reading and math are compounded by a weak education management system and poor infrastructure. Since school fees were abolished in 2002, Zambia has made significant gains toward ensuring near universal access to primary education to the approximately 3.2 million children enrolled in grades 1-7,1 yet learners reading levels remain low, particularly in community schools which serve 20 percent of the most economically disadvantaged children. Only 30 percent of children who enter the school system manage to complete the education cycle through grade 12. The poor quality of education has exacerbated inequities for girls and disadvantaged children in general.

Given the learner demographics, teacher qualifications, and resource challenges facing most

Zambian schools, improving learning outcomes is critical.

The Early Grade Reading Assessment (EGRA) was first administered in Zambia in 2011-2012 to students in several targeted provinces but not on a national scale. The results revealed that 90% of children in grade 2 and 70% in grade 3 were not able to read one word in the local Zambian language. Further, the Zambia Grade 5 National Assessment Survey is conducted every two years and has consistently shown learning achievement below 40th percentile in both English and Zambian languages. In addition, the Southern African Consortium for Monitoring Education

Quality (SACMEQ) of 2010 indicated that among the Grade 6 learners that were tested in reading, only 27.4 percent were able to read at a basic competency level. In 2014, USAID contracted Research Triangle Institute (RTI) to work with the Examination Council of Zambia

(ECZ) to conduct the first national EGRA baseline, which revealed that Zambian students were still performing below average. The previous EGRA report is provided.

Republic of Zambia, Ministry of Education, Science, Vocational Training and Early Education: Educational

Statistical Bulletin 2015

In order to address the low literacy levels among learners, the MOGE developed a national literacy strategy of teaching reading in local languages and also introduced these local languages as a medium of instruction in grades one through four. The MOGE shifted the focus from access to quality learning with assessment data providing the critical role of tracking the progress of student learning over time. Timely information on student reading achievement and its implication for future learning provides a basis for dialogue to inform policy and decision making among education leaders.

USAID/Zambia Education efforts over the past 5 years have focused on early grade reading interventions to confront these challenges. These programs included STEP-Up, Read to

Succeed, Time to Learn, Zambia EdData and other Government to Government agreements.

These USAID funded activities were designed to Improve Educational Achievement in Reading by

2017 through (1) strengthening the systems of the Ministry of General Education, (2) improving the performance of public and community schools, (3) increasing equitable access to education for all students and, (4) mitigating the impact of HIV/AIDS on the education sector. The efforts over the last five years, as well as the future efforts are rooted in the development hypothesis that if time spent on reading instruction is sufficient; if children are taught to read in a language that they speak and understand; if there are enough and relevant teaching and learning materials in the seven local languages; if teachers are well-trained, mentored, and coached to teach and formatively assess reading acquisition in local languages using the appropriate approaches; if the education system is adequately equipped to test early grade reading skills and report and analyze results; if the MOGE has the appropriate capacity to implement and manage resources at all levels; and if families and communities are involved in the education of their children; then early grade students will be able to read with fluency and comprehension at grade level. The development hypothesis is depicted in the results framework below:

C.4 SUCCESS

Success will be measured by the completion of two Early Grade Reading Assessments (EGRA) for children in Grade 2 measuring reading levels, during the three years and six months

IR3.1 Educational Achievement in Reading

Improved

SIR 3.1.1

Ministry of

Education

Systems

Strengthened

SIR 3.1.2 Public and Community

School

Performance

Improved

SIR 3.1.3

Equitable Access to Education

Increased

SIR 3.1.4

HIV/AIDS

Impact on

Education

Mitigated

Figure 1: Results Framework contract period. Upon completion of each EGRA, proper dissemination and learning will be critical for the wider USAID and stakeholder audience in Zambia to apply lessons learned and trends to future programming.

C.5 KEY STAKEHOLDERS

Implementation of this activity will require coordination and collaboration with key stakeholders. The following list provides key stakeholders and their respective roles and responsibilities within the USAID Education Data Activity:

Table 1: Key Stakeholder Listing

Stakeholder Role

USAID/Zambia USAID/Zambia will have oversight of the performance of the

Contract. USAID/Zambia will serve as a liaison between the

Contractor and MOGE, and will obtain necessary approvals to facilitate data collection and other EGRA related activities in the provinces, districts and schools. The Education Office will work with the Contractor to disseminate EGRA findings to various stakeholders.

Ministry of General Education

(MOGE)

The MOGE will be involved and consulted during activity planning and implementation to ensure alignment with policy provisions and institutionalization of EGRA related activities in the education system. The Contractor must work closely with the MOGE’s Directorate of Planning and Information

Management and Directorate of Curriculum and Standards, to inform, develop and/or finalize the content of instruments, and ensure alignment of school level data with the Education

Management Information System (EMIS). The MOGE will coordinate provincial, district, zonal and school level activities.

In addition, the directorates will collaborate with the

Contractor to disseminate EGRA findings to the lower levels of the education system.

Examination Council of Zambia

(ECZ)

As an institution mandated to conduct Zambian national assessments and examinations, the ECZ will be involved at all stages of EGRA activities; i.e., from the preparation of surveys to dissemination of EGRA findings both at national and regional levels. Working in close collaboration with the Contractor, the

ECZ’s role will involve, but not be limited to, selecting representative samples for the EGRA, developing and/or finalize the content of instruments, identifying data collectors/assessors;

assist to develop, pre-test and finalize survey instruments;

collect and analyze assessment data and organize meetings/fora to disseminate EGRA findings to relevant stakeholders.

Provincial Education Offices and

District Education Board Secretary

Office, The Provincial and District Education Offices will work with the

Contractor to coordinate sub national level activities, which include: preparation for EGRA surveys, identification of assessors and providing necessary authorizations, supporting data collection processes and regional dissemination workshops. As implementing units of education programs, the provincial and district levels will use EGRA findings for planning, management, monitoring and evaluation of education projects.

Target Schools These are the intended primary beneficiaries of the Let’s Read

Zambia activity from which the EGRA sample will be drawn. In addition to providing USAID, MOGE and other education stakeholders with information on reading outcomes, ultimately the EGRA findings are expected to inform decision making and management processes at the school level.

Cooperating Partners (Irish Aid, JICA, UNICEF, World Bank & China

Fund-in-Trust/UNESCO )

These include other donors providing funds for education activities independently but aligned with the objectives of the

Let’s Read Zambia activity. In addition, donors such as JICA could potentially fund the mathematics component of Early

Grade Assessments to complement the USAID funded reading assessment. Therefore, collaboration with these stakeholders will be necessary to ensure alignment of activities.

C.6 LINK TO OTHER PROGRAMS

This activity will conduct the baseline and midline surveys for the separate bilateral USAID Let's

Read Zambia Early Grade Reading activity. Let's Read Zambia will work in approximately 4000 schools of which 1,281 have Early Childhood Education annexes or units, across five Provinces

(Lusaka, Eastern, Muchinga, North Western and Southern Province). The five Provinces have approximately 400 zones, made up of a cluster of approximately 10 schools within each zone.

The school focus will be on Pre-Grades through Grade 3. Let's Read Zambia’s focus will be to provide the target school students with the opportunity to learn to read grade level text with comprehension in the seven local official languages of instruction as described in Table 2.

Table 2: Official Languages of Instruction in the Five target Provinces:

Province Official Language of

Instruction

Eastern Chinyanja

Lusaka Chinyanja2

Muchinga Icibemba

North Western Lunda, Luvale, Kiikaonde

Southern Chitonga3

C.7 SCOPE

The USAID Education Data activity will focus on two main areas:

1. EGRA: The EGRA will be administered at the baseline and midline for the upcoming

USAID Early Grade Reading activity. Through the EGRA, USAID will be able to determine if the USAID Early Grade Reading activity is helping to ensure that children are reading at grade level.

To ensure that the assessment planning and implementation are sustained by the government, the contractor must work closely with ECZ. The Contractor must build on the work done by RTI, the Read to Succeed and Time to Learn activities to strengthen the ECZ’s capacity to conduct future EGRA.

2. Research services: Four distinct research studies will be conducted to inform USAID and MOGE programming needs. This is discussed in Section.C.8.

C.8 TASKS

The tasks under this activity are to:

1. Complete a language mapping exercise;

2. Complete preparation for the Early Grade Reading Assessment (EGRA);

3. Complete pilot assessment;

4. Finalize EGRA instruments;

5. Train EGRA Assessors;

6. Conduct EGRAs;

7. Complete data entry, analysis, and build ECZ’s capacity;

8. Disseminate EGRA results to key stakeholders; and

9. Conduct Research Studies.

Due to the creation of new districts some parts of Lusaka province such as Chirundu and Shibuyunji and Kafue rural are predominantly Chitonga speaking Parts of Kazungula district is predominantly Silozi speaking

Task 1: Language Mapping Exercise:

The Contractor must ascertain whether the predominant language of instruction within each

District, and if possible school level, is as depicted in Table 2, under C.6 of this section, . The purpose of this exercise is to inform the Contractor to be able to administer the EGRA in the

GRZ designated language per Province; however, also have the ability to administer the EGRA in a different GRZ designated language(s) if the Contractor identifies that is the language of instruction in the particular school/district. The Contractor must verify GRZ designated language(s) of instruction at the school level through the Baseline and adjust the language mapping based on findings. If the Contractor finds instances where the language of instruction is outside of the GRZ designated languages, they must validate and document such findings through the baseline.

Task 2: Preparation of Draft Instruments to conduct EGRA:

Through the Research Triangle Institute (RTS) implemented EdData II activity, a framework/toolkit (link provided in Section J) was developed to serve as a resource on EGRA instruments development and adaptation for Zambia. Building on this work, the Contractor shall review and if necessary, update the EGRA Framework to ensure that best practices and lessons learned at baseline and midline are incorporated.

The Contractor must prepare for the EGRA by working with the USAID, MOGE, and ECZ in order to define the components listed below. The coordination with the three stakeholders will result in a draft standardized instrument for the contractor to carry out the EGRA within the GRZ designated languages of instruction.

EGRA Components:

Component 1: For the Learners (Child):

Letter-sound Identification: The ability to produce the sound of a letter fluently that is presented in written form. It is timed to one minute.

Syllable naming fluency: The ability to identify the most commonly occurring syllables in a particular language. It is timed to one minute.

Invented (nonword) Reading: ability to “decode” unfamiliar words. The “nonwords” are decodable, are ‘legal’ words, and are fictional. It is timed to one minute.

Oral reading fluency: The ability to quickly and accurately read connected text on a page with the passage written for an end of grade 2 difficulty level. It is timed to one minute.

Reading Comprehension: The ability to orally respond to both literal and inferential questions about the Oral Reading Fluency passage read. This component does not have a time limit.

Listening comprehension: The ability to comprehend an orally presented story and provide an oral response to question asked. This component does not have a time limit.

English listening comprehension: The ability to comprehend an orally presented story and provide an oral response to question asked. This component does not have a time limit.

English Vocabulary: The ability to identify body parts, objects in the environment, and simple prepositions presented in English. This component does not have a time limit.

Component 2: Snapshot for School Management Effectiveness (SSME)

This component of the EGRA will produce a comprehensive picture of a school’s learning environment, and when the results from multiple schools in a region are compared, it becomes possible to account for differences in school performance. Using information generated through the SSME component, school, district, provincial and/or national administrators can learn what is occurring in schools and classrooms and assess how to make their schools more effective.

Among others, the complementary instruments will target education administrations including, head teachers, teachers and zonal personnel.

School Profile:

Along with the components above, the contractor must collect and update the following data on each school where the EGRA is conducted:

1. Name of school

2. Name of head teacher

3. Contact details

4. Location (Province, district and zone)

5. Type of school (community or public, with or without Early Childhood Education(ECE)

6. GRZ Designated Language of Instruction

7. Education Management Information System (EMIS) number

8. Total number of students (disaggregate by sex, grade)

9. Total number of teachers (disaggregated by sex/trained by Let's Read Zambia)

10. Literacy teaching and learning materials (Pupil: book ratio)

USAID Indicator:

ES.1-1 Percent of learners who demonstrate reading fluency and comprehension of grade level text at the end of grade 2 with USG assistance

USAID may require additional indicators that will be identified through the life of the award, if applicable.

Task 3: Pilot Assessment

A small scale assessment will be undertaken with the draft EGRA for all GRZ designated languages of instruction. No more than 15 schools in total that represent the GRZ designated language of instruction will be assessed in this Task, on average two schools per language. The pilot assessment will allow the contractor to understand if the draft EGRA needs updating and/or if the components need further improvement to properly measure reading performance at Grade 2.

Task 4: Finalization of the Instruments to conduct EGRA

The Contractor must provide the feedback from the pilot assessment to USAID, MOGE, and

ECZ and apply lessons learned to finalize the standardized instruments to be used in the EGRA.

Task 5: Assessor Training

Each EGRA must be administered by an assessor not associated with the school. The

Contractor's approach to administering the EGRA must ensure the child’s comfort and provide an environment which is conducive to obtaining needed information under each component.

The Contractor is responsible for identifying and providing rigorous training to the assessors.

A key consideration in the recruitment of assessors must be familiarity with the local language of instruction.

Task 6: Conduct EGRA

The Contractor must conduct EGRAs in an electronic format via tablets or other electronic means. The contractor is responsible for random sampling for 25 percent of approximately

4000 schools, per 400 zone, per 48 district, per 5 target provinces. At a minimum, the

Contractor's random student sampling must consist of 20 students per school. Prior to conducting the EGRA, the contractor must provide the random school listing and schedule to

USAID to obtain proper approvals from the MOGE.

Each EGRA must be administered in the GRZ designated language of instruction for the province. However, if it is determined that multiple GRZ designated languages of instruction are in use, the contractor must be able to administer the EGRA in each GRZ designated language and document instances where the Provincial GRZ designated language of instruction do not match the school level language of instruction in use.

Each EGRA must be completed by November 30th. The following schedule is developed for the anticipated EGRAs:

Base-Line The Base-Line is to be conducted within six months after award. If the contract is awarded after November, then the baseline must be completed within one year after award prior to the beginning of the academic year in which the Let’s Read Zambia classroom instructional intervention is implemented

Mid-Line The Mid-Line will be timed after two years of implementation under Let's Read Zambia.

The Contractor must work with ECZ to help transition the ability to lead, manage, and execute the EGRA over time. USAID expects that the Contractor must be the lead on the Base-Line, allowing ECZ to shadow all aspects of the EGRA. For the Mid-Line, USAID expects the

Contractor to identify specific tasks for ECZ to complete in conjunction with the contractor that make up approximately 50% of the EGRA. Through the proposed transition approach, the

Contractor in consultation with USAID will determine the roles and responsibilities to gradually transfer to the ECZ. The Contractor must immediately notify USAID if ECZ is unable to take on any responsibilities and ensure that all EGRAs are completed. The ultimate responsibility for timely completion of EGRA and data analysis is with the Contractor.

Task 7: Data Entry, Analysis, and ECZ Capacity Building

The Contractor must analyze and consolidate data to produce the final EGRA reports. The

Contractor must involve ECZ and MOGE experts to foster sustainability of the project and build capacity within the Ministry for data collection and analysis. All data collected must be stored on the Development Data Library (DDL) and the USAID Secondary Analysis of Results

Tracking (SART Ed) Portal https://usaideducationdata.org/sart/.

ECZ will specifically need capacity building in data collection and analysis, conducting large scale assessments and data management. The Contractor must develop a transition/capacity building strategy clearly outlining the proposed transfer of roles and responsibilities over the three years and six months contract period. The plan must document what the ECZ would require to conduct similar surveys in future and, must include a tentative implementation plan and estimated costs for routine and nationally representative EGRA.

Task 8: Data Dissemination

After each EGRA, the Contractor must disseminate findings and analysis to a larger stakeholder audience. The dissemination meetings will address the findings as they relate to the intended learning outcomes of students and the proposed performance targets/indicators for Let's Read

Zambia. USAID expects the same approach as stated under Task 6; whereby, the contractor must work to have ECZ become the lead in disseminations and only provide administrative and technical oversight towards the final EGRA dissemination meeting.

Base-Line Provincial/District/Zone Dissemination

Mid-Line National/Provincial/District

Dissemination

Task 9: Research Studies

The Contractor will provide four (4) distinct research projects. The Contractor must work closely with USAID and the MOGE to determine which of the areas of research listed below and/or any other study areas identified which would be most beneficial to address programming needs and provide the best support to address the problems faced by the MOGE. The contractor must then help the MOGE conduct this research. This will include both desk review and primary research on topics that directly impact learner outcomes in early grade reading in

Zambia. Potential areas of research would include:

https://usaideducationdata.org/sart/

● The value-added of students using technology to improve learners’ performance;

● A comparison of the impact on student learning with teacher use of tablets with lesson plans and coaching guidance vs. student use of tablets; and

● A study that follows Kindergarten students over time to determine the impact of emergent literacy skills interventions.

● A study on the threshold when students should transition from Language one (L1) to

Language two (L2) or from the local languages to English.

C.9 GENDER

The USAID/Zambia’s gender analysis identified that gender bias and inequitable learning conditions in Zambian schools continue to undermine educational achievement for both boys and girls. Poor educational outcomes exacerbate gender inequalities and often leave young

Zambians with few skills of the ability to participate in the economy. The national EGRA baseline conducted in November 2014 identified that there are only minimal achievement differences between boys and girls in the early grades. It is more cost-effective to address the causes for the divergence in learning outcomes in the early grades when these differences are small than to wait for girls to drop out, repeat, or fail their school exams. It is expected that the

Contractor must assess the extent to which Let's Read Zambia ensures that learning gains benefit both males and females and in all language groups. Let's Read Zambia will help reduce perceived learning gaps along gender and linguistic groups. The report must describe ways in which Let's Read Zambia is working to narrow gaps between boys and girls and among the different languages of instruction and any success or challenges encountered. Data must also be disaggregated by sex, type of school, geographic location and other categories required by

USAID.

C.10 ENVIRONMENTAL COMPLIANCE

The Foreign Assistance Act of 1961, as amended, Section 117 requires that the impact of

USAID’s activities on the environment be considered and that USAID include environmental sustainability as a central consideration in designing and carrying out its development programs.

This mandate is codified in Federal Regulations (22 CFR 216) and in USAID’s Automated

Directives System (ADS) Parts 201.5.10g and 204 (http://inside.usaid.gov/ADS/200/204.pdf), which, in part, require that the potential environmental impacts of USAID-financed activities are identified prior to a final decision to proceed and that appropriate environmental safeguards are adopted for all activities.

The USAID Zambia Education Initial Environmental Examination (IEE) has a Categorical

Exclusion for the Intermediate Result (611-IR3.1) Educational Achievement in Reading

Improved. The IR is supported by four sub-IR: (3.1.1) Ministry of Education Science, Vocational

Training and Early Education (MESVTEE) systems strengthened; (3.1.2) Public and community school performance increased; (3.1.3) Equitable access to education increased; and (3.1.4)

HIV/AIDS impact on education mitigated.

A Categorical Exclusion is recommended for activities under sub-IR’s 3.1.1, 3.1.2, 3.1.3, 3.1.4 under Program Element 3.2.1 Basic Education, involving education, technical assistance, or http://inside.usaid.gov/ADS/200/204.pdf training programs (as per 22 CFR 216.2(c)(2)(i)); analyses, studies, academic or research workshops or meetings (as per 22 CFR 216.2(c)(2)(iii)); document and information transfers (as per 22 CFR 216.2(c)(2)(v)); studies, projects or programs intended to develop the capability of recipient countries to engage in development planning (as per 22 CFR 216.2(c)(2)(xiv)); and regulatory and policy related activities that have no physical interventions and no direct effects on the environment (as per 22 CFR 216.2(c)(1)(i)).

Activities under Program Element 3.2.1 Basic Education, involve education, technical assistance, or training programs; analyses, studies, academic or research workshops or meetings;

document and information transfers; studies, projects or programs intended to develop the capability of intended beneficiaries or local governance structures to engage in development planning; and regulatory and policy related activities that have no physical interventions and no direct effects on the environment. These activities are recommended for a Categorical

Exclusion under 22 CFR 216.2(c)(1)(i) and 22 CFR 216.2(c)(2)(i), (iii), (viii) and (xiv). See above citations

The Contractor must comply with the stated environmental regulations. The program managers and the Contractor must take measures to ensure such information sharing does not promote practices that would negatively impact the environment and that the information sharing sessions themselves have negligible impacts on the environment.

C.11 SUSTAINABILITY

To ensure sustainability, the Contractor must build the institutional and human capacity of the

ECZ to independently conduct national early grade reading assessments and provide timely information about student reading competencies. The Contractor must train ECZ Officers in developing EGRA contextual questionnaires, data collection and analysis using agreed upon software, large scale assessment and data management. Institutionalized capacity building is in line with the USAID Education Project’s two main hypotheses: (1) improving the overall quality of primary education requires improving basic reading skills in early grades; and (2) facilitating host country leadership of the development agenda is critical for long-term sustainability and true sector-wide impact.4

[END OF SECTION C]

USAID/Zambia Education Project Appraisal Document 2013