Attachment_3-Draft_SBEP_PMP.pdf

SINDH BASIC EDUCATION

PROGRAM

PERFORMANCE MANAGEMENT PLAN

NOVEMBER 28, 2012

This publication was produced for review by the United States Agency for International

Development (USAID). It was prepared by Management Systems International (MSI) under the Monitoring and Evaluation Program (MEP).

shumayun Typewritten Text

ATTACHMENT 3

shumayun Typewritten Text shumayun Typewritten Text shumayun Typewritten Text shumayun Typewritten Text

RFP SOL-391-13-000005

Amendment No.1 Dated February 05, 2013 shumayun Typewritten Text shumayun Typewritten Text shumayun Typewritten Text shumayun Typewritten Text shumayun Typewritten Text shumayun Typewritten Text shumayun Typewritten Text

SINDH BASIC EDUCATION

PROGRAM

PERFORMANCE MANAGEMENT PLAN

Contracted Under No. GS-23F-8012H and Order No. AID-391-M-11-00001

Monitoring and Evaluation Program (MEP)

DISCLAIMER

The author’s views expressed in this publication do not necessarily reflect the views of the

United States Agency for International Development or the United States Government.

i

TABLE OF CONTENTS

Acronyms ..................................................................................................................................................... ii

Introduction Project Background Definitions Overview

Section 2: SBEP Logic Model

Section 3: SBEP PMP Table

Section 4: Performance Indicator Tracking Table

Section 5: Performance Indicator Reference Sheets Goal: Increased and Sustained Student Enrollment in Primary, Middle and Secondary

Schools in Targeted Geographic Locations of Sindh 28 Outcome 2.1: Improved student access to supplementary reading material 45 Outcome 2.2: Improved Teacher Competencies 46 Outcome 2.2.1: Skills strengthened among teachers/educators/teaching assistants 47 Outcome 2.3: Improved early grade reading and math assessment 47 Outcome 2.4: Enhanced participation of parents and communities in support of promoting reading and numeracy among school children 48 Outcome 3: Increased Girls Enrollment 50 Outcome 3.1-5.1: Increased community engagement and mobilization 51 Outcome 4: Improved Health Status of Children 51 Outcome 4.1: Improved health practices in targeted communities 52 Outcome 4.1.1: Increased community knowledge of child health and nutrition 53 Outcome 4.1.2: Increased child access to health services 54 Outcome 5: Effective Civil Society for Oversight, Engagement and Advocacy 54 Outcome 5: Effective Civil Society for Oversight, Engagement and Advocacy 55 Outcome 6: Improved Management Capacity at Provincial and District Levels within the

Department of Education 56 Outcome 6.1: Improved financial and procurement management in the PMIU 57 Outcome 6.1: Improved financial and procurement management in the PMIU 58 Outcome 6.2: Improved Education Administration 59 Outputs 62

Section 6: Remaining Tasks to Complete PMP

Annexes Annex 1: Linkage to USAID/Pakistan Results Framework Annex 2: Selection Criteria for School Reconstruction Annex 3: MEP Statement of Work for PMP Development ii

ACRONYMS

ADS Automated Directives System

ASER Annual Status of Education Report

ASP Assessment and Strengthening Program

CDC Centers for Disease Control

EGRA Early Grade Reading Assessment

EMIS Education Management Information System

GoS Government of Sindh

GPI Gender Parity Index

IQRA Improving the Quality of Reading Activity

IR Intermediate Result

MEP Monitoring and Evaluation Program

MSI Management Systems International, Inc.

PIRLS Progress in International Reading Literacy Study

PISA OECD Programme for International Student Assessment

PMP Performance Management Plan

PTA Parent-Teacher Association

SACMEQ Southern and Eastern Consortium for Monitoring Educational Quality

SEMIS Sindh Education Management Information System

SBEP Sindh Basic Education Program

SMC School Management Committee

SODIS Solar Disinfection System

SOW Statement of Work

TA Technical Assistance

TLM Teaching and Learning Materials

TOR Terms of Reference

UNESCO United Nations Educational, Scientific and Cultural Organization

UNICEF United Nations Children’s Fund

USAID United States Agency for International Development

USG United States Government

INTRODUCTION

Project Background

On September 21, 2011, the U.S. Agency for International Development (USAID) signed an

Activity Agreement with the Government of Sindh (GoS) for the Sindh Basic Education

Program (SBEP). SBEP will focus on increasing and sustaining student enrolment in primary, middle and secondary schools in seven districts of Sindh, plus Karachi, by developing a school environment conducive to teaching and learning. This transformation will be achieved through the following components: (1) construction of schools affected by

2010 floods; (2) support to GoS policy reforms to merge, consolidate and upgrade schools through construction of schools; (3) improvement in early grade reading in primary schools;

(4) community mobilization, with a focus on increasing girls’ enrolment and improving the nutritional status of children; and (5) technical assistance to the Department of Education.

According to page 8 of the Notification to the Economic Affairs Division of USAID’s project with Sindh, SBEP will provide technical assistance to the GoS in three areas: (1) financial and procurement management; (2) engineering oversight and design; and (3) education administration.

SBEP will be implemented in seven underserved districts of northern Sindh (Kashmore, Jacobabad, Sukkur, Kambar Shahdadkot, Khairpur, Dadu and Larkana) and Karachi’s five towns (Kemari, Lyari, Orangi, Qasim and Gadap). The life-of-program funding is estimated at $165 million ($155 million from USAID and $10 million from the GoS) over a five-year period (September 2011 to September 2016)

Definitions

This section provides standard definitions for some of the key terms used in this Performance

Management Plan (PMP).

Counterfactual: What would have occurred had a specific program or project not been implemented. The counterfactual in impact evaluations answers the question, “what difference has the program/project made?”

Disaggregate: Disaggregated data are data for a single indicator that are broken apart, analyzed and reported at the level of a group or category. For example, for an indicator disaggregated by sex, data would be analyzed and reported for men and women (or boys and girls) separately.

Evaluation: As defined by USAID, evaluation “is the systematic collection and analysis of information about the characteristics and outcomes of programs and projects as a basis for judgments to improve effectiveness, and/or inform decisions about current and future programming. Evaluation is distinct from assessment, which may be designed to examine country or sector context to inform project design, or an informal review of projects. The

These are the dates stipulated in the PC-1. The draft SBEP PMP from USAID shows targets for 2011 through 2015.

purpose of evaluations is to ensure accountability to stakeholders and learn to improve effectiveness.”

Input: Inputs are “the human, financial, organizational, and community resources a program has available to direct toward doing the work.”

Impact evaluation: As defined by USAID, impact evaluations “measure the change in a development outcome that is attributable to a defined intervention. Impact evaluations are based on models of cause and effect and require a credible and rigorously defined counterfactual to control for factors other than the intervention that might account for the observed change.”

Logic model: A logic model is a visual map that shows how a project or program intends to use its resources (“inputs,” including staff, funds, etc.) to achieve outputs (direct results of activities), which lead to outcomes (higher-level results) to the long-term program/project goal. It also identifies assumptions inherent in the program design, as well as factors outside the control of the program/project that could affect its success.

Outcome: Outcomes “are the specific changes in program participants’ behavior, knowledge, skills, status and level of functioning. Short-term outcomes should be attainable within

1-3 years, while longer-term outcomes should be achievable within a 4-6 year timeframe. The logical progression from short-term to long-term outcomes should be reflected in impact occurring within about 7-10 years.”

Output: Outputs “are the direct products of program activities and may include types, levels and targets of services to be delivered by the program.”

Performance evaluation: As defined by USAID, performance evaluations “often incorporate before-after comparisons, but generally lack a rigorously defined counterfactual.

Performance evaluations focus on descriptive and normative questions:

What a particular project or program has achieved;

How it is being implemented;

How it is perceived and valued;

Whether expected results are occurring; and

Other questions pertinent to program design, management and operational decision making.”

Performance indicator: As defined by USAID, a performance indicator is a “particular characteristic or dimension used to measure intended changes defined by a (logic model) or a project. Performance indicators are the basis for observing progress and measuring actual results compared to expected results. Performance indicators help answer how or if (a project/program) is progressing towards its (goal), rather than why such progress is or is not being made (evaluations respond to the question of why or why not progress is being made).

The PMP must include performance indicators for all levels of a (logic model).”

USAID Automated Directives System (ADS) section 203.3.2

Introduction to Logic Models, the W. K. Kellogg Foundation

Ibid.

USAID ADS section 203.3.2.1

USAID ADS section 200.6

Performance baseline: As defined by USAID, a performance baseline is the “value of a performance indicator before the implementation of USAID-supported projects or activities that contribute to the achievement of the relevant result.”

Performance management: As defined by USAID, “performance management is the systematic process of monitoring the achievements of program operations; collecting and analyzing performance information to track progress toward planned results; using performance information and evaluations to influence… decision-making and resource allocation; and communicating results achieved, or not attained, to advance organizational learning….”

Performance Management Plan (PMP): As defined by USAID, a PMP is “a tool to plan and manage the process of monitoring, evaluating, and reporting progress towards achieving a” goal.

PMPs normally include a hierarchy of results that a project, program, or office intends to achieve over the short, medium and long term (i.e., a logic model or equivalent);

and a list of performance indicators to measure progress towards each result in the hierarchy.

The purpose of a PMP, as its name suggests, is to support performance management. The

PMP should be updated with new performance data as they are collected.

“During project implementation, (programs/projects) may need to change or drop PMP performance indicators. For example, there may be changes in program priorities or budgetary decisions that affect the scope/geographic focus of the (goal) which would require the use of indicators different from those originally selected. Indicators may need to be adjusted, if the indicators prove to be unsuitable, for example, if the effort and cost needed to collect them become excessive. Indicators may also be added as lessons are learned about project dynamics during implementation and through evaluations. (Programs/projects) should note that changing performance indicators can compromise the comparability of performance data over time. This is a particularly critical issue for indicators at the (goal) or outcome level. It should be clear that the caution about changing indicators refers to all levels of results in a (logic model)… It is expected that over the life-cycle of a project, different types of indicators will be more useful at different times: in the early stages, output indicators are likely to be the primary source of performance information, while at later stages outcome and impact indicators will become more important. The full set of indicators should be specified at the outset and even indicators that are subsequently dropped–because they are unsatisfactory or no longer used because their targets have been met–should be retained for reference in PMP records… (The program/project) should note the reason(s) for the change, along with final values for all old indicators and baseline values for any new indicators.”

Any changes to the SBEP PMP should be agreed upon by both the GoS and USAID.

Performance target: As defined by USAID, the performance target is “specific, planned level of result to be achieved within an explicit timeframe with a given level of resources.”

Targets should be set for each indicator in the PMP. The PMP should include an end-of-

USAID ADS section 200.6

USAID ADS section 200.6

USAID ADS section 203.3.4.7 program target for each indicator and may include interim targets, as well. Annual targets are required for all indicators reported to USAID. “Targets should be ambitious, but achievable given USAID (and other donor) inputs. (Programs/projects) should be willing to be held accountable for achieving their targets. On the other hand, targets that are set too low are also not useful for management and reporting purposes.”

Overview

This PMP is intended to assist USAID to manage the SBEP program and make informed decisions based upon high quality performance data.

The overall goal of SEP, as defined in the PC-1 and Activity Agreement, is “Increased and sustained student enrollment in primary, middle and secondary schools in targeted geographic locations in Sindh.” USAID and the GoS intend to accomplish this goal through five components:

Component 1: Construction of schools affected by 2010 floods.

Component 2: Support to Government of Sindh policy reforms to merge, consolidate and upgrade schools through construction of schools.

Component 3: Improvement in early grade reading in primary schools.

Component 4: Community mobilization, with a focus on increasing girls’ enrollment and improving nutritional status of children.

Component 5: Technical assistance to the Department of Education.

The USAID-funded Monitoring and Evaluation Program (MEP), implemented by

Management Systems International (MSI) developed this PMP in consultation with the

USAID/Pakistan Mission. MEP primarily used existing documentation, including the most recent version of the final PC-1, its annexes, the Activity Agreement with the GoS, and the

RFA for the Improving the Quality of Reading Activity (IQRA) to understand the program and modify the PMP to ensure that it is relevant for managing for results and meets the criteria established by USAID. MEP selected provisional indicators for measuring results based upon the documentation provided. These indicators will be used to establish the baseline for the program, and at the outcome level, help set the stage for a final outcome or impact evaluation.

Please note that the PMP table lists a number of indicators that are to be reported to USAID.

Extra care must be taken to adhere to the indicator definitions and ensure the collection and reporting of high quality data for these indicators.

The remainder of this document is organized as follows: Section 2 presents the project’s logic model, while Section 3 describes the performance indicators to be used to measure progress towards each of the results in the logic model. Section 4 presents a table of baselines and targets for all of the indicators in the PMP. For indicators that feed into the Mission’s DO-level PMP, the PIRS are almost the same as the Mission’s PIRS, with small changes made to complete them and make them relevant to SBEP. Section 5 includes the Performance

Indicator Reference Sheets (PIRS) for all of the indicators in this PMP. The USAID program

USAID ADS section 203.3.4.5 office has recently made changes to the PIRS format and these changes are still the draft. In cases where USAID Education indicators are included in the SBEP PMP, the revised draft

PIRS template is used in the documents. However, for rest of the indicators, the previously approved PIRS template is used. Section 6 lists the additional tasks to be done to complete the PMP and manage performance through the life of SEP. Annexes contain a description of how this PMP contributes to the USAID/Pakistan Mission PMP, the school selection criteria developed for SBEP, and the statement of work (SOW) for developing this PMP.

Please note than in all cases in which an indicator asks for a percentage, SBEP should report not only the percentage, but also the numerator and denominator.

SECTION 2: SBEP LOGIC MODEL

Outputs

Activities Participation

Outcomes

Immediate Short-term Medium-term

Outcome 1:

Improved

Educational

Facilities

Outcome 2:

Improved Reading

Skills for Children in Primary Grades

Improved teacher competencies

Improved early grade reading and math assessment

Improved student access to supplementary reading material

Enhanced participation of parents and communities in support of promoting reading and numeracy among school children

Outcome 5: Effective Civil

Society for Oversight, Engagement and Advocacy

Outcome 3:

Increased Girls’

Enrollment

Outcome 4:

Improved Health

Status of Children

Outcome 6: Improved

Management Capacity at

Provincial and District

Levels within the

Department of Education

Improved financial and procurement management in the PMIU

Improved Education

Administration

Long-term Goal: Increased and sustained student enrollment in primary, middle and secondary schools in targeted geographic locations in the Sindh province

Increased community engagement and mobilization

Improved health and hygiene practices in targeted communities

Schools constructed

Classrooms built or repaired

Teaching and learning materials (TLM) provided

Teachers/educators/teaching assistants who successfully completed in-service training or received intensive coaching or mentoring with USAID support

Children in targeted districts reached by health programs

People trained in child health and nutrition

Children in targeted districts who received micronutrients

Links created to local basic health units

Administrators and officials successfully trained with USAID support

Standard learning assessments supported by USAID

SMCs or similar ‘school’ governance structures supported

Skills strengthened among teachers/educators/teaching assistants

Laws, policies, regulations, or guidelines developed or modified to improve primary grade reading programs or increase equitable access

SMCs strengthened

Increased community knowledge of child health and nutrition

Increased child access to health services

Assumptions

Funds are sufficient and their release is on time

There are no overwhelming capacity issues in the Government of Sindh

Child-friendly school premises motivate parents to send children to school

Construction of schools promotes education policy reforms

Political will to undertake reforms

No major setbacks with devolution of education services

Provincial ministries will develop improved learning assessment systems

Security challenges will not greatly affect accessibility for implementation and supervision

Weak public sector capacity at all levels for planning, financial management, and governance

There is no litigation by stakeholders

Stability of Pakistan

Continuing US Government partnership and assistance to education in Pakistan

Active participation of local and provincial government officials and communities

External Factors

Natural disasters

There is no litigation by stakeholders

There are no delays in procurement of supplies due to shortage or other factors

Political willingness of GOP and provincial governments to undertake education sector reforms in a decentralized system

Interest of the private sector to partner with USAID to improve the quality of teaching and learning

Security challenges and public perception about USG civilian assistance in Pakistan

Weak public sector capacity at all levels in planning, financial management, governance, and assessment

SECTION 3: SBEP PMP TABLE

Indicator Name, Disaggregates Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator

Program Goal: Increased and Sustained Student Enrollment in Primary, Middle and Secondary Schools in Targeted Geographic Locations in Sindh

1. Percent of primary school age population that attends school (net primary enrollment rate) disaggregated by school type

(girls, boys, mixed), school level (primary, elementary or secondary), and district.

Definition: The enrollment rate is the percentage of the primary school-age population that attends school. The percentage is calculated by dividing the number of students enrolled in school divided by the number of all school-aged children in target districts.

School records in

SEMIS

SEMIS generated report

Once a year MEP (year 1), GOS thereafter

The net enrollment rate is a direct indicator of the extent to which SBEP has increased student enrollment in targeted schools.

No

2. Percent change in primary, middle, and secondary school completion rates in USG-assisted schools, disaggregated by sex, rural/urban, school type

(boys, girls, mixed), school level (primary, middle, secondary), and district

Definition: The number of students enrolled in the first grade of each of primary, middle, and secondary school who remain in school through the end of the last grade of each of those schools. The indicator measures the percent change in the completion rates from one year to the next year, which is calculated by finding the difference between the completion rate baseline (first year of USG intervention) with current year completion rate divided by the completion rate baseline and multiplied by 100.

SEMIS

SEMIS generated report

Once a year MEP (year 1), GOS thereafter

This indicator is a measure of the extent to which the

IP has sustained student enrollment. The overall intent of this project is to ensure that children complete primary, middle and secondary schools.

New PIRS template used

Yes

3. Change in number of learners enrolled in primary, elementary and secondary

USG assisted schools or equivalent USG assisted non-school based settings, disaggregated by sex, rural/urban, school type

(boys, girls, mixed), school level (primary, middle, secondary), district

Definition: Two standard indicators combine to form this indicator: 3.2.1-14 (number of learners enrolled in primary schools and/or equivalent non-school settings with USG support); and 3.2.1-15 (number of learners enrolled in secondary schools or equivalent non-school based settings with USG support).

The definitions for each of these contributing indicators are listed below.

3.2.1-14: Number of learners formally enrolled in primary schools or equivalent non-school based settings for the purpose of

School records in

SEMIS

SEMIS generated report or record review

Once a year MEP (year 1), GOS thereafter

USAID funds used to improve primary, elementary and secondary education

SI (see

PIRS)

Disaggregates Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator acquiring academic basic education skills or knowledge. This number may include learners receiving educational radio and/or TV programming.

4. Percent change in school drop-out rates in USG-assisted schools, disaggregated by sex, rural/urban, school type

(boys, girls, mixed), school level (primary, middle, secondary), district

Definition: Percentage change in the dropout rate of students in target areas as demonstrated by the combined total pre & post intervention headcounts from all targeted schools. Percentage change will be calculated by determining the difference between the total combined pre- and post-intervention data and dividing it by the total combined pre-intervention headcount from all targeted schools and multiplying by 100.

School records in

SEMIS

SEMIS generate report or record review

Once a year MEP (year 1), GOS thereafter

The dropout rate measures the success of the school in retaining its student population during a normal school year.

Outcome 1: Improved Educational Facilities

1. Number and proportion of

USG supported schools or learning spaces meeting criteria for safe schools programs, disaggregated by rural/urban, school type

(boys, girls, mixed), district

Definition: Safe school criteria include:

Schools or learning spaces are secure environments for children/learners; provide a stimulating and supportive environment for children; are built on existing structures and capacities within a community; use a fully participatory approach for the design and implementation; provide or support integrated services and programs; are inclusive and non-discriminatory. Proportion of schools will be calculated against the total targeted number of schools Proportions will range from 0 to

100%.

School

Constructio n Reports and school records

Portfolio reviews;

records verified by field visits of M&E data collectors

Once a year Sindh

Reading

Program

(TBD) and/or

Community

Mobilization

(TBD)

For improved quality of teaching, facilities such as staff areas and research space are important factors.

SI

2 .Number of schools consolidated, disaggregated by (boys, girls, mixed) rural/urban, school level and district

Definition: SBEP GOS intends to promote a policy reform “to consolidate several small schools that exist in a village or neighborhood into a single facility under a single cadre of administrative personnel.

District education offices

Report review –

Direct Observation

Once a year GOS One of the policy reforms targeted to improve the educational facilities in

Sindh is the consolidation of schools under a streamlined administration.

This indicator measures the extent to which this reform is implemented.

No

Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator

3. Percent change in funding for school operational and maintenance costs in SBEP-assisted districts, disaggregated by type (boys, girls, mixed) and rural/urban

Definition: The amount of funding for school operational and maintenance costs in the current year’s budget minus the amount of funding for the same costs in last year’s budget, divided by the amount of funding for these costs in last year’s budget. To count, operational and maintenance costs must have their own line item(s) in budgets for the

SBEP-targeted schools.

Sindh school budgets from targeted districts

Review of relevant budget documents

Once a year GOS GoS has initiated school-specific budgets, which are expected to “provide resources for maintenance and supplies” (PC-1).

SBEP is supposed to support this policy initiative. Therefore, the indicator measures the effectiveness of this policy support in actually increasing the amount of funds available to help maintain and operate targeted schools.

No

4. Percent change in pupil – teacher ratio, disaggregated by rural/urban, school level

(primary, elementary, secondary), and district

Definition: The pupil-teacher ratio is the measure of the average number of pupils per teacher in a given school year.

School records in

SEMIS,

SBEP-

targeted schools

SEMIS data, complemented by site visits to a sample of SBEP-targeted schools and telephone calls

(where feasible) to other SBEP-targeted schools;

other secondary sources

Annual formal data collection with informal checks during the year

MEP (year 1), Sindh

Reading

Program

(TBD)

thereafter

According to the PC-1, “The fourth policy reform is to hire additional female teachers based on merit and qualifications.” The fifth reform includes the placement of “school-specific teachers.”

Therefore, this indicator is a direct measure of the efficacy of these policy reforms.

No

5. Teacher absenteeism rate, disaggregated by rural/urban, district, school level, sex

Definition: The percent of teachers absent from SBEP-targeted schools. “Percent of teachers absent” is measured by dividing the number of teachers absent in SBEP-supported schools by the total number of teachers listed on the payroll for those schools.

SEMIS,

SBEP-

targeted schools

SEMIS data, complemented by site visits to a sample of SBEP-targeted schools and telephone calls

(where feasible) to other SBEP-targeted schools;

other secondary sources

Annual formal data collection, with informal checks during the year.

MEP (year 1), Sindh

Reading

Program

(TBD) and/or

Community

Mobilization

(TBD)

thereafter

One of SBEP’s targeted policy reforms under

Component 2 is “the adoption of school-specific budgets and the placement of school-specific teachers, which will…reduce teacher absenteeism.” In addition, greater involvement of parents and SMCs in schools should help improve teacher attendance. Therefore, this indicator measures the

Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator extent to which SBEP reduces teacher absenteeism - an important indicator of the extent to which Sindh education governance has improved.

Outcome 2: Improved Reading and Numeracy for Children in Primary Grades

1. Proportion of students who, by the end of two grades of primary school, demonstrate that they can read and understand the meaning of grade level text., disaggregated by sex, (boys, girls, mixed), province

Definition: Proportion of learners who attain the specified threshold at the end of two grades of primary schooling, the beginning of the third year of primary schooling, or the equivalent levels of accelerated learning programs. Students and learners in formal and non-formal education programs should be included. Measures of the indicator will be determined in consultation with the GoS and informed by national (or provincial, if applicable) curriculum standards, and by international experience and by standards set out in “read to learn”. e.g. demonstrating satisfactory levels of comprehension as measured by comprehension questions on grade 2 texts, or reading a country-determined number of words correct per minute.

EGRA Under the Sindh

Reading Program, EGRA

(assessments and interview protocols) will be developed and pilot tested in 2012.

Baseline will also be collected for the end of the 2011/12 school year using a sample population of 4,050 students

(treatment group only).

Once a year Sindh

Reading

Program

(TBD)

This indicator is an outcome measure of the highest order for Goal 1 of the Education Strategy, which calls for improved reading skills for 100 million primary school children. Learning to read is the foundation for future learning in all subjects and at all levels of education.

Improvements in basic literacy also have been shown to translate to higher national economic growth rates as well as to other population-level benefits.

2. Proportion and number of students who by the end of primary cycle, are able to read and demonstrate understanding as defined by a country curriculum, standards or national experts, disaggregated by sex, Definition: Proportion of students and learners who attain a country-defined threshold indicative of reading comprehension at the end of the primary cycle, as defined through the (national or sub-national) curriculum. Students and learners in formal and non-formal education programs focusing on reading should be included. The language of assessment will be determined by country policies and curriculum.

EGRA Data will be generated through written reading comprehension assessments carried out at the end of primary school, except in cases of very low performance where oral assessment may be needed.

The sampling approach must

Once a year Sindh

Reading

Program

(TBD)

This indicator measures the extent to which improvement is occurring.

This indicator is more useful for measuring impact – the extent to which graduates of primary school are better prepared to read in secondary school than they were before began.

Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator generate data representative of the districts

3. Proportion and number of students who by the end of primary cycle, are able to use numbers and demonstrate understanding of numeracy as defined by a country curriculum, standards or national experts, disaggregated by sex, Definition: Proportion of students and learners who attain a country-defined threshold indicative of numeracy proficiency at the end of the primary cycle, as defined through the (national or sub-national) curriculum. Students and learners in formal and non-formal education programs focusing on numeracy should be included. The language of assessment will be determined by country policies and curriculum. Proportion is obtained by dividing the number of students and learners with numeracy proficiency at the end of the primary cycle (or non-formal equivalent) by the total number of students and learners at the end of the primary cycle

(or non-formal equivalent).

EGMA Under Sindh

Reading Program, EGRA and EGMA

(assessments and interview protocols) will be developed and pilot tested in 2012.

Baseline will also be collected for the end of the 2011/12 school year using a sample population.

Once a year Sindh

Reading

Program

(TBD)

This indicator measures the extent to which improvement is occurring.

This indicator is more useful for measuring impact – the extent to which graduates of primary school are better prepared in numeracy in secondary school than they were before began.

No

Outcome 2.1: Improved Student Access to Supplementary Reading Material

1. Percent of teachers correctly using SBEP-provided supplementary reading materials disaggregated by sex, rural/urban, school type

(boys, girls, mixed), district

Definition: The number of teachers trained who use their SBEP-provided reading material and/or use their newly acquired skills to develop their own supplementary materials divided by the total number of teachers trained in these areas under SBEP.

SBEP-

trained teachers

Sample survey Once a year Sindh

Reading

Program

(TBD)

Simply having materials available is not sufficient for improving student learning. This indicator measures the extent to which teachers are actually using/developing materials as they were meant to be used.

No

Outcome 2.2: Improved Teacher Competencies

1. Percent of USAID-trained teachers observed to be employing improved teaching techniques, disaggregated by sex, rural/urban, district

Definition: Of the total number of teachers trained by SBEP in improved techniques for teaching reading and math skills, the percent of teachers observed to be using those techniques in the classroom. Specifically, teachers will develop an extensive repertoire of student-centered learning pedagogies, SBEP-teachers trained

Observation of trained teachers teaching in their classrooms, complemented where useful by interviews with

Once a year Sindh

Reading

Program

(TBD)

Part of the SBEP TA to the

GoS is supposed to focus on improving the SEMIS system. This indicator measures the extent to which that system is

Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator amending classroom instruction based on classroom populations and student learning styles.

teachers and/or students (see Data

Limitations).

expanded.

Outcome 2.2.1: Skills Strengthened Among Teachers / Educators / Teaching Assistants

1. Percent of teachers/educators/teaching assistants with increased skills, disaggregated by sex, rural/urban, school type

(boys, girls, mixed), district

Definition: The number of teachers/educators/teaching assistants trained who score higher on post-training tests than they did on pre-training tests, divided by the total number of teachers/educators/teaching assistants trained by SBEP.

Training

Reports

Pre- and post-training tests

Quarterly Sindh

Reading

Program

(TBD)

A key strategy for this project is to train teachers in improved reading instruction. This indicator measures the extent to which the training has resulted in improved knowledge/skills.

No

Outcome 2.3: Improved Early Grade Reading and Math Assessment

1. Percent of targeted districts implementing the

EGRA and/or EGMA, disaggregated by urban/rural

Definition: Of the districts in which Sindh

Reading Program introduces the EGRA and/or EGMA, the percent of districts that are actually implementing the test(s) on an annual basis.

DOE

district officials

Record review, interviews with selected DOE officials

Once a year Sindh

Reading

Program

(TBD)

Sindh Reading Program intends to introduce EGRA and/or EGMA (as appropriate) to the DOE to improve the frequency and quality of student assessment, thereby providing data which schools can use to improve teaching and, thereby, student learning outcomes.

This indicator measures the extent to which districts actually adopt the

EGRA/EGMA tests.

No

Outcome 2.4: Enhanced Participation of Parents and Communities in Support of Promoting Reading and Numeracy Among School Children

1. Number of parents participating in SMCs and/or other community group, disaggregated by sex, rural/urban, school type

(boys, girls, mixed), district

Definition: The number of parents of children in Sindh Reading Program-assisted schools who actively participate in SMCs and/or other community group. “Active participation” to be defined by Sindh Reading Program implementing partner, in consultation with

SMCs / community groups/scho ols/Minutes of the meetings of

Review of SMC / community groups’ meeting notes, and interviews leaders/ members where useful

Quarterly Sindh

Reading

Program

(TBD) and/or

Community

Mobilization

One of the key methods

Sindh Reading Program intends to use to increase involvement of parents and communities in promoting child learning is reviving and strengthening the

Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator

USAID.

SMCs (TBD) SMCs. This indicator measures the extent to which participation in

SMCs does, in fact, increase.

THIS INDICATOR

NEEDS TO BE

DISCUSSED WITH IP

BEFORE MOVING

FORWARD

2. Number of functioning

Executive Committees through interventions of community Kachehries

Definition: Kachehries are groups of male and female community members who engaged in fortnightly reading activities. Functioning is defined as those groups meeting on a regular basis during the previous quarter.

Executive committees/ village level committees

Minutes/notes from executive committees

Quarterly Sindh

Reading

Program

(TBD)

This indicator measures the extent to which communities embrace reading enrichment activities.

No

Outcome 3: Increased Girls Enrollment

1. Gender Parity Index, disaggregated by district, rural/urban

Definition: Ratio of female to male gross enrollment ratios. The “gross enrollment rate” is defined as a nation’s total enrollment in a specific level of education, regardless of age, expressed as a percentage of the population in the official age group corresponding to this level of education.

School records in

SEMIS

Review USAID implementing partner reports

Once a year MEP/ USAID implementing partner

SBEP focuses especially on increasing girls’ enrollment by encouraging parents to send their daughters to school. The GPI measures the extent to which SBEP does so.

No

Outcome 3.1-5.1: Increased Community Engagement and Mobilization

1. Percent of school management committees

(SMCs), CSOs or other committees that are implementing school development plans (or achieving TBD) disaggregated by rural/urban, school type (boys, girls, TBD: Community Mobilization

Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator mixed), and district

Outcome 4: Improved Health Status of Children

1. Global Acute Malnutrition rate in children aged 6 -108 months in targeted districts, disaggregated by sex, rural/urban, school level, district, school type (boys, girls, mixed)

Outcome 4.1: Improved Health and Hygiene Practices in Targeted Communities

1. Percent of households in target areas following child health guidelines for their children, disaggregated by rural/urban

Outcome 4.1.1: Increased Community Knowledge of Child Health And Nutrition

1. Percent of trained community members that demonstrate increased knowledge of child health and nutrition, by rural/urban and district

Outcome 4.1.2: Increased Child Access to Health Services

1. Number of targeted schools that receive visits from local basic health units, disaggregated by rural/urban, mixed), and district

Outcome 5: Effective Civil Society for Oversight, Engagement and Advocacy

Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator

1. Percent of parents who express confidence in the school system, disaggregated by sex and district

Definition: Number of parents who report being “confident or very confident” when asked “How confident are you in the school system where your child is being education?”

divided by the total number of respondents.

Parents involved in

SBEP-

supported school districts

Random sample survey

Once a year MEP Currently, there is a lack of confidence in the quality of public education systems.

As reforms take place and there is improvement in the governance of basic education (meaning that communities have more input into decision-making), then confidence should increase.

Yes

2. Number of USAID-assisted CSOs demonstrating improvement on advocacy index, disaggregated by district

Outcome 6: Improved Management Capacity at Provincial and District Levels within the Department of Education

1. Number of institutional weaknesses

Definition: The number of weaknesses during the current assessment minus the number of weaknesses identified in the baseline assessment. “Institutional weaknesses” are agreed-upon institutional weaknesses identified through the pre-award assessment of the PMIU and recorded on the risk mitigation framework.

PMIU Institutional

Assessment

Once a year USAID

Assessment and

Strengthening

Program

(ASP)

The PC-1 states that the

ASP will strengthen institutional weaknesses identified in the pre-award assessment. This indicator measures the extent to which ASP is successful in remedying these weaknesses and, thereby, strengthening the capacity of the PMIU.

No

2. Active use of GIS by the

GoS

Definition: The number of times the GoS uses or updates the GIS system transferred from

IMMAP and paid for with USG funds.

Active Use is defined as the entry of geo-coded data, the printing of maps or other visual data or queries at least once a quarter

SEMIS Verification of use of GIS through maps, data entry or log-books

Quarterly PMIU SBEP is funding IMMAP to develop a GIS system that includes school related data and coordinates for the target areas. This system is expected to be transferred to the GoS as part of

SEMIS. This indicator measures the extent to which the system is being used for planning, Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator reporting and decision making.

Outcome 6.1: Improved Financial and Procurement Management in PMIU

1. Percent of school construction contracts procured competitively

Definition: the number of contracts for school construction in SBEP-assisted areas of Sindh that meet standard competitive procurement requirements divided by the total number of contracts for school construction in SBEP-assisted areas of Sindh.

PMIU Inspection of procurement documentation, complemented where useful by interviews with

PMIU procurement officials

Once a year TBD One of the core areas of

SBEP’s assistance to the

PMIU, according to the

Notification to the

Economic Affairs Division of USAID’s project with

Sindh, is strengthening the procurement system. This indicator measures an important aspect of procurement strengthening.

No

2. Percent change in audit observations as compared to last year

Definition: Difference of current and last year’s financial management (FM) related internal audit observations divided by number of FM related audit observations of the last year.

Internal

Audit reports

Review of internal audit reports

Once a Year PMIU ASP’s mandate is to develop systems and procedures for SBEP in areas of financial management and procurement management, along with other operational areas. ASP will also provide on job technical assistance to

SBEP PMIU. This indicator measures implementation and continued compliance of financial management processes and procedures developed for SBEP.

No

Outcome 6.2: Improved Education Administration

1. Percent of education administrators and officials trained who are using the training in their work, disaggregated by sex, level

Definition: Number of education officials

(public or private) or administrators of education programs, funds or institutions trained by SBEP in areas such as management and quality assurance for improving reading

Officials trained

Interviews with trained officials or post-training surveys

Every six months

USAID

Implementing

Partner

A key part of improving education administration is capacity building. This indicator measures the extent to which SBEP-

Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator of government skills at the primary level, or supporting the re-integration of conflict-affected youth into the formal or non-formal education systems, who confirm that they are using the training in their work 6 and 12 months following their training; divided by the total number of

SBEP-trained administrators and officials.

provided training is effective.

2. Number of District

Education Plans developed and approved through USG assistance, disaggregated by district

Definition: District Education Plan — planning that contains data about the educational needs of schools that is objective and uniform and where an agreement is reached on the priorities for future possible funding. The contractor provides technical assistance to local government officials to synthesize existing educational data and then facilitate with appropriate stakeholders to develop district-level strategic plans for the education sector and identify gaps in educational performance and prioritize programs for future funding. The proposed plan promotes good governance and management in education.

District

Education

Offices

Records collected from the District

Education Offices

Once a year SBEP District

Education

Office/PMIU/

GOS

USAID training supports capacity building for host country education administrators and officials and their institutions.

New PIRS template used

Yes

3. Number of school-specific teachers placed, disaggregated by sex, rural/urban, district, school type (boys, girls, mixed)

Definition: “School-specific” means that a teacher is assigned to a specific school when hired, rather than to a school district or other organization. The name or other identification of the school to which a teacher is assigned must appear in the teacher’s contract or other hiring papers. “Placed” means “assigned.” If such papers do not exist, the teacher’s name and assigned school must be documented somehow, perhaps in a school report or district education office report.

This is a CONTEXT Indicator

Targeted schools, districts

Review of district education hiring plans and reports and/or teacher hiring papers

Once a year District

Education

Offices/PMIU

/GOS

According to the PC-1, “The fifth policy reform is the… placement of school-specific teachers, which will… reduce teacher absenteeism.” Therefore, this indicator is a direct measure of the extent to which the targeted policy reform is, in fact, resulting in school-specific teacher placement. It is an indication that the TA has resulted in improved GoS management.

Definition, Unit of Measure

Data

Source

Data Collection

Method

Data

Collection

Frequency

Data

Collection

Responsibilit y

Rationale

USAID

Indicator

Outputs

1. Number of schools constructed, disaggregated by type (boys, girls, mixed), rural/urban, district, rationale (flood/merger/ consolidation/upgrading)

Definition: The number of schools constructed, furnished, and equipped by

SBEP in target districts in Northern Sindh and towns in Karachi. To count against this indicator, a school must be completely built, though does not need to contain furniture.

USAID

Constructio n project implementi ng partner reports

Report review Once a year GoS One of the main parts of the SBEP project is to construct schools to replace those swept away during the 2010 floods. This indicator measures the extent to which SBEP fulfills this planned activity.

No

2. Number of classrooms built or repaired with USG assistance, disaggregated by rural/urban, district, school level (primary, elementary, secondary), school type

(boys, girls, mixed)

Definition: Classrooms are safe and secure spaces in which organized group learning takes place. Classrooms range from environmentally-appropriate, roofed structures without walls, to traditional four-walled structures with a roof and windows.

Individual classrooms should be counted if a whole classroom block is built or repaired.

This indicator does not include temporary classrooms (such as tents, open spaces set aside for instruction). This indicator measures only the classrooms constructed and/or renovated by USAID implementing partners in target districts or flood-affected areas. A classroom should be counted only once during the life of the project.

USAID

Constructio n project implementi ng partner reports

Field visits by data collectors to check construction and rehabilitation numbers

Quarterly GoS Classrooms of acceptable quality are an essential component of education, making instruction possible and encouraging parents to send their children to school. Adequate school buildings positively impact access to education.

3. Number of textbooks and other teaching and learning materials provided with USG assistance, disaggregated by rural/urban, province, direct/indirect…