Attachment-10_Final_Report.doc

Revitalizing, Innovating, Strengthening Education (RISE)

In Earthquake Affected Areas of Pakistan Cooperative Agreement No. 391-A-00-06-01080-00

FINAL EVALUATION

Randy L. Hatfield, PhD, Team Leader

Usman Ali, Education Specialist

June 2010

This program evaluation is made possible by the support of the American people through the United States Agency for International Development (USAID.) The contents of this evaluation are the sole responsibility of its authors, and do not necessarily reflect the views of USAID or the United States Government.

Table of Contents 4Acknowledgements

5List of Acronyms

7Executive Summary

10Background

12Approach and Methodology

14EFFECTIVENESS

31EFFICIENCY

36RELEVANCE

39RECOMMENDATIONS

Acknowledgements

For their co-operation, courtesy and their participation in the evaluation, we are deeply indebted to parents, communities and teachers of Mansehra, Bagh, Poonch and Muzaffarabad and to RISE and USAID officials met in these districts as well as in Islamabad. Gratitude is also extended to Officials of the Government including the district education departments and ERRA/SERRA for their full cooperation and participation in various meetings during the evaluation.

Particular acknowledgement and thanks also go to Marilee Kane and Nazlee Sardar, Officers of USAID/Pakistan for their technical direction and for facilitating the evaluation. Also, special acknowledgement to Dr. Naeem Butt, Program Director of RISE and the RISE senior staff for their participation in the self assessment. Although RISE staff considered themselves to be a team regardless of organizational affiliation, the support of development partners such as Sungi Development Foundation, Sarhad Rural Support Program and the National Rural Support Program (NRSP) are appreciated for their support and guidance throughout the evaluation team’s field visits.

We are also very much indebted to the RISE Core team for their full support and participation in the field visits, observations and meetings. They provided critical data when requested, accompanied us to site visits in all the districts and even provided translation support where required. Finally, our appreciation is extended to Riaz Khan, Raja Qadeer Khan, Saleema Malik, Mansoor Ahmed Maqshbandi, Farhat Abbas Warriach and Mrs. Tanveer Latif for a splendidly organized and enjoyable evaluation program.

List of Acronyms

ABES

Adult Basic Education Society

ACR

Annual Confidential Report

ADB

Asian Development Bank

ADEO

Assistant District Education Officer

ADO

Assistant District Officer (Mansehra only)

ADS

Automated Directives System

AEO

Assistant Education Officer (Bagh only)

AIR

American Institute for Research

AJK

Azad Jammu & Kashmir

ASC

Annual School Senses

CO

Community Organization

COP

Chief of Party

CUP

Community Uplift Program

CWS

Church World Service

DCRD

Directorate for Curriculum, Research and Development

DCTE

Directorate for Curriculum and Teacher Education

DCTE

Devolution Trust for Community Empowerment

DEE

Directorate of Education Extension

DDEO

Deputy District Education Officer (Bagh only, equivalent to DO in Mansehra)

DEO

District Education Officer (Bagh only, equivalent to EDO in Mansehra)

DEP

District Education Plan

DFID

Department for International Development

DoE

Department of Education

DQA

Data Quality Assessment

DRU

District Reconstruction Unit

DTW

Districts That Work

EMIS

Education Management Information System

ERRA

Earthquake Reconstruction and Rehabilitation Authority

GBHS

Government Boys High School

GBPS

Government Boys Primary School

GGMS

Government Girls Middle School

GGPS

Government Girls Primary School

GOP

Government of Pakistan

GPS

Government Primary School

I-LED

Improving Livelihoods and Enterprise Development

INGO

International Non Government Organization

IRC

International Rescue Committee

KEAS

Kashmir Educational Assessment System

KPK

Khyber Pakhtoonkhaw

LRC

Learning Resource Center

LSO

Local Support Organization

M&E

Monitoring & Evaluation

MERA

Muhammadan Educational Research Associates

MIED

Mountain Institute for Education Development

NCHD

National Commission for Human Development

NEAS

National Educational Assessment System

NGO

Non Government Organization

NRSP

National Rural Support Program

NWFP

North West Frontier Province

NWFP

North West Frontier Province

OFDA

Office of Foreign Disaster Assistance

PDF

Professional Development Forums

PDP

Professional Development Plans

PRIDE

Primary Health Care Revitalization, Integration and Decentralization in Earthquake Affected Areas

PTC

Parents Teacher Council (used in Mansehra starting in August 2007)

RISE

Revitalizing, Innovating, Strengthening Education

RSP

Rural Support Program

RSPN

Rural Support Program Network

S/A

Satisfactory/Advance

SC

Steering Committee

SERRA

State Earthquake Reconstruction and Rehabilitation Authority

SIP

School Improvement Plan

SMC

School Management Committee (Used in AJ&K)

SRSP

Sarhad Rural Support Program

TQI

Teacher Quality Index

UC

Union Council

UNESCO

United Nations Educational, Scientific and Cultural Organization

UNICEF

United Nations Children’s Fund

US/NI

Unsatisfactory/Need Improvement

USAID

United States Agency for International Development

VO

Village Organization

VSO

Village Social Organization

Executive Summary On October 8, 2005 Pakistan suffered a massive earthquake of 7.6 magnitude that took more than 73,000 lives and displaced 2.8 million people from their homes. To help strengthen the Education system in the affected area, USAID signed a four-year cooperative agreement with the American Institutes for Research (AIR) on July 25, 2006. USAID/Pakistan awarded $13 million for the Revitalizing, Innovating, Strengthening Education (RISE) Project. USAID’s Intermediate Result 8.4 became the major goal of the RISE project: Education System Strengthened. At the outset of the program, RISE was tasked to meet this goal as measured through two indicators of improved student achievement and teacher attendance. It was further anticipated that the RISE Project would be considered successful if improvement in these two areas had been achieved and the project had met the goals/targets for each component. The project initially targeted the Mansehra District of Khyber Pakhtunkwa and Bagh District of Azad Jammu & Kashmir (AJK). In the first quarter of 2008, USAID added an additional $10 million to expand the project scope to include Districts Poonch and Muzaffarabad in AJK. RISE engaged the following components:

RISE Project Components

1) Improved education management;

2) Improved quality of classroom teaching; and

3) Increased community participation in school management An evaluation of the RISE Project was commissioned by USAID in May 2010 to determine in general terms:

· Did RISE meet benchmarked activities as outlined in the original cooperative agreement and presented in annual work plans?

· Are the strategies and approaches for strengthening and enhancing delivery of education effective, and do they reflect state-of-the-art education knowledge and approaches?

· What are major gaps, if any, in terms of the response of the project to strengthening education in the earthquake zone?

· What components of RISE should USAID continue to support through the core USAID program?

The evaluation approach included a desk review of project documents and field visits including 28 focus group meetings with 247 stakeholders across the four districts. This report analyzes the RISE Project and its implementation by AIR and partners in terms of effectiveness, efficiency and relevance. The major findings and recommendations are summarized below.

FINDINGS

(1) Did RISE meet benchmarked activities as outlined in the original cooperative agreement and presented in annual work plans?

Yes. The project has achieved its intended goals of reduced teacher absenteeism and improved student achievement. In addition, RISE met all of the project targets included in the cooperative agreement and included in the PMP. The project inputs are identified in short

· Over 160 education managers were trained on improved school planning and management techniques

· Over 10,000 teachers trained

· Teacher training curriculum was student-centered and based on Pakistan’s national curriculum

· 2,300 SMC/PTC were established/activated and trained Further, the approach of the program was flexible to address the social problems of the stakeholders after the harrowing affects of the earthquake while not compromising on quality of technical assistance through teacher training instruction, materials and monitoring and research techniques.

(2) Are the strategies and approaches for strengthening and enhancing delivery of education effective, and do they reflect state-of-the-art education knowledge and approaches?

Yes. Component #1 served to provide training and support for education officers whose offices and files, in many cases had been destroyed. Component #2 provided badly needed materials and fresh teacher training approaches that worked with teachers and supported activity based learning regardless of the environments, e.g. shelterless, tents, or badly deteriorated school buildings. Component #3 enabled communities to understand their roles and rights in managing the affairs and planning for their respective schools and provided them with seed funds for improving their schools based on collaborative planning. Under this component, communities demonstrated their commitment to education through financial and in-kind donations to their respective schools in all the districts. As the respondents in this evaluation unanimously asserted, the Project truly revitalized, innovated and strengthened education in the earthquake-affected areas for the people of Pakistan.

(3) What are major gaps, if any, in terms of the response of the project to strengthening education in the earthquake zone?

According to both the RISE self-assessment and the findings of this evaluation, RISE was limited and could not target all the schools, teachers and SMCs/PTCs in the districts, nor did RISE cover all subjects taught in the schools. RISE provided support in the three subjects that teachers identified as most difficult: English, mathematics, and science. However, the middle and high school curricula are extensive and include many subjects. This limitation prevented other subject teachers from attending the trainings and developing their skills not only in their subjects but also in active learning and student-centered techniques. If RISE had focused on all the schools, teachers, subjects, and SMCs/PTCs, the education system could have been strengthened further.

Secondly, RISE provided small grants to only 50 percent of the schools. In a post earthquake context, grants to all the schools would have helped in meeting the immediate needs of schools. Additionally, the small grants activity is a rallying point for the SMCs/PTCs and helps build their confidence and skills in project implementation and momentum for their continued and active support to their schools.

(4) What components of RISE should USAID continue to support through the core USAID program?

The overall design of RISE and its three components were both relevant and appropriate to the education needs in the aftermath of the earthquake and aligned with reconstruction programming in the affected areas. The main thrusts of the project are highly consistent with the Ministry of Education’s policies past and present and with ERRA’s education strategy. Therefore, all three components should be included in future USAID support to the same geographical areas.

Infrastructure is still a very big need of the government and should be included as a new component. A total of 2,800 schools are to be constructed in AJK. Only 400 have been completed and 1,400 are in the pipeline for construction and 1,000 have yet to be started. For the completion of these 1,400 schools the funding requirement is Rs. 27 billion. The Government has only allocated Rs. 10 billion. With the shortage of the government budget, there is a need to help construct more schools. Decision needs to be made with ERRA/SERRA to prioritize infrastructure and soft component projects.

Although USAID is currently supporting an independent school construction program in the earthquake affected areas, the Mission should also consider tying both soft and hard components together in future programs across Pakistan.

Background On October 8, 2005 Pakistan suffered a massive earthquake of 7.6 magnitude that took more than 73,000 lives and displaced 2.8 million people from their homes. USAID’s Office of Foreign Disaster Assistance (OFDA) was mobilized within 48 hours and provided more than $68.7 million in humanitarian assistance to Pakistan in response to the earthquake. A Special Objective Grant Agreement for USAID’s four-year, $200 million dollar Earthquake Reconstruction program was signed with the Government of Pakistan (GOP) in January 2006. In addition to rebuilding schools and health facilities, the Reconstruction program was designed to strengthen health and education systems and restore livelihoods for communities and individuals in selected areas of Khyber Pakhtunkwa (formerly NWFP) and AJK.

The Joint Asian Development Bank (ADB)-World Bank damage assessment of November 15, 2005 reported that due to the earthquake, more than 18,000 students and 853 teachers and staff lost their lives in NWFP and AJK. In addition, over 5,000 educational institutions were damaged and required reconstruction. The Earthquake Rehabilitation and Reconstruction Authority’s (ERRA) vision for building back better includes the following elements a) better construction designs; b) provision of better physical facilities; c) better classroom support to improve students’ learning achievement level; d) Rationalized provision of schools; e) better and improved ownership and management of the educational institutions through parents’ participation and; f) better social service delivery.

To help strengthen the education system in the affected area, USAID signed a cooperative agreement with the American Institutes for Research (AIR) on July 25, 2006. USAID/Pakistan awarded $13 million for the Revitalizing, Innovating, Strengthening Education (RISE) Project to address the soft component of ERRA’s strategy for rebuilding the education system in the earthquake-affected areas. USAID’s Intermediate Result 8.4 became the major goal of the RISE project: Education System Strengthened. At the outset of the program, RISE was tasked to meet this goal as measured through two indicators of improved student achievement and teacher attendance. It was further anticipated that the RISE Project would be considered successful if improvement in these two areas had been achieved and the project had met the goals/targets for each component. RISE is comprised of the following project components: 1) Improved education management; 2) Improved quality of classroom teaching; and 3) Increased community participation in school management.

The project initially targeted the Mansehra District of Khyber Pakhtunkwa and Bagh District of Azad Jammu & Kashmir (AJK). In the first quarter of 2008, USAID awarded an additional $10 million to expand the project scope to include Districts Poonch and Muzaffarabad in AJK.

RISE was a four-year initiative (2006-2010) that ended July 2010. USAID has commissioned this independent final evaluation of the project in compliance with its due diligence for project monitoring and evaluation.

Approach and Methodology The main objectives of the evaluation included an assessment of the performance against objectives of the RISE Project, a review of the achievements, challenges and lessons learned, a comparison of RISE self-assessment with the assessment of the independent evaluation and finally the development of some strategic recommendations that would benefit the project in the future should continuation occur (See the Terms of Reference in Appendix A).

The Evaluation was carried out between May 13 and June 10, 2010 and was conducted by a team consisting of an international consultant who served as the team leader and a Pakistani education specialist. The methodology used for undertaking the evaluation included:

· A review of documents submitted by USAID/RISE including quarterly reports, performance monitoring plans, annual reports, annual implementation plans and other related documents;

· Interviews with principal stakeholders including staff from RISE core team, RISE and Project Partner field officers, government, USAID and communities, teachers and students;

· Non-participant observation of relevant processes including lessons and other school activities;

· Interactions with teachers, head teachers, children, parents, PTCs/SMC members and others involved at school level;

· Discussion with ERRA/SERRA government officials in Mansehra and Muzaffarabad Districts respectively.

(See Appendix B for Workplan)

The evaluation began with a briefing by the RISE core team in Islamabad on the program including an overview of program goals, objectives and major features of each component as briefed by component Directors. Afterwards, a total of 28 focus groups were conducted with approximately 247 stakeholders of the RISE Project across the four- targeted districts. Minutes have been drafted for each focus group/meeting (See Appendix C for the list of focus group meetings).

The evaluation team also prepared a self-assessment tool at the outset of the review and asked the RISE team to self-administer the instrument that was prepared in the format of a powerpoint to encourage greater participation of RISE officers (See Appendix D for self-assessment instrument). On May 20, 2010, RISE convened senior field, partner, and Islamabad-based staff for a one-day meeting to respond to the self-evaluation questions as a team. Approximately 20 RISE officers worked together to prepare the collective response (See Appendix E for the response to the self-assessment & list of participants).

The report is organized below according to three major areas: Effectiveness (measuring impact), Efficiency (reflection on management and implementation performance) and Relevance (recording successes, lessons learned and recommendations for future programming).

EFFECTIVENESS

Overall Aim: To establish whether RISE achieved intended goals and results.

Stakeholders including government officials, teachers and members of school management committees including parents interviewed were unanimous in their feedback that the RISE Project had helped to change their own understanding and behavior in a positive way for improving the education system in their respective districts. Education managers acknowledged RISE support as the means for their improved abilities for planning, budgeting and monitoring teachers and schools. Teachers demonstrated positive results of the RISE Project by being more regular in attendance to their classrooms and practicing new techniques such as activity-based learning. School management committees expressed their newly recognized role and “rights” by increased participation in school improvement planning and monitoring.

The overarching goal of the RISE project is an Education System Strengthened (USAID’s Intermediate Result 8.4). At the outset of the program, RISE was tasked to meet this goal as measured through two indicators of improved student achievement and teacher attendance. It was further anticipated that the RISE Project would be considered successful if improvement in these two areas had been achieved and the project had met the goals/targets for each component. (See diagram below)

Source: RISE PMP

According to RISE monitoring and reporting documentation, the project has achieved its intended goals of reduced teacher absenteeism and improved student achievement. In addition, RISE reports achievement of meeting the project targets as listed below.

RISE Targets and Achievements

IR 8.4 Education system strengthened

trained in education management 161 district education mangers received training on management.

(135 males & 26 females)

1.1

1.2

100% managers trained on EMIS 69 Computers delivered to education departments in 4 districts.

161 managers received training on EMIS. (135 male & 26 female)

1.3

Improved implementation of coherent in-service teacher training system

4 steering committees formed and meetings held (One Steering Committee meeting per quarter)

Steering Committees formed and notified by the respective Education Departments in each district.

35 Steering Committee meetings held.

Component 2

Quality of classroom teaching improved

320 schools will participate in Subh-e-Nau activities every year.

600 science kits &1200 Cluster stationery distributed with some science material.

197 subh-e-Nau events celebrated in which 1,294 schools participated. (618 boys & 676 girls)

Cluster stationery along with science equipment distributed in 1,200 schools while science kits have yet to be distributed.

2.1

Train 10,000 teachers

10,000 teachers trained

Total 10,316 (5,346 male & 4,970 female) teachers trained.

361 Master Trainer trained in 4 districts.

2.2

10% of schools providing peer engagement activities.

78% primary schools and 73% of middle school teachers are attending monthly cluster follow-up meetings.

382 out of 959 mature clusters conducted monthly cluster meetings on their own.

Component 3

2,300 SMCs/PTCs established/reactivated.

3.1

3.2

Improvement Plan and implemented at least one activity from their SIP.

1,427 SMCs/PTCs (726 male & 701 female) implemented at least one component of their SIP.

3.3

Establish and implement small grant program to support SMCs/PTCs.

1,146 (50% of total trained) PTCs/SMCs received small grants;

1,146 (50% of total trained) PTC/SMCs implemented Small Grant Program.

1,146 (542 boys & 604 girls) SMCs/PTCs were issued small grants in all the four districts.

1,145 (542 boys & 603 girls) SMCs/PTCs completed their small grants projects in all four districts.

Each component is described in the following section along with associated issues and challenges. Afterwards, the two key quality indicators of the RISE project are analyzed in detail.

Component 1: Improved Education Management

In terms of achievement against planned targets, all targets were achieved under this component in all four districts. RISE began work on this component with emphasis of supporting Education Managers within their respective offices. A rapid needs assessment survey (Management Checklist) was developed and a workshop conducted with government managers to assess their needs against six management areas: 1) Financial Management and Budgeting; 2) Personnel Management; 3) Planning and Development; 4) School Supervision and Support; 5) SMC/PTC Support; and 6) Teacher Training. Interviews were also conducted. The baseline assessment was conducted on the basis of “perceived needs” of education managers with the purpose of eventually providing support to education managers to address their specific requirements and needs.

Results of the baseline compared with later surveys indicate marked improvement in education managers’ skills and abilities. The approach to training was not theoretical but was considered by most managers to be more practical. Training required participants to visualize an ideal education system and then to answer the question of how each manager could improve the current system. Education Managers were also encouraged to implement their assignments more effectively but to recognize the shared ownership and responsibility of management with communities and teachers.

Capacity building workshops were organized on financial management and budgeting for education managers in all the target districts. These workshops introduced concepts and practical activities related to financial management, needs-based budgeting, advocacy for fund raising and timely disbursement of available funds. During the workshops, education managers made a commitment to develop needs-based budgets for at least 10-15 schools in each circle of the district every quarter.

One important achievement of these workshops is the engagement of government personnel as resource persons. Accounts and planning experts from the Department of Finance & Planning, Accountant General’s offices, and district education offices conducted sessions in all the target districts. These resource persons were also involved in the needs assessment and planning for the workshops.

RISE also provided computers and workshops for EMIS staff on data entry, cleaning and generation of query-based reports from the Access Database. Moreover, 699 lead data collectors were also trained in the use of data collection tool and data collection procedures for Annual School Census (ASC) in AJ&K RISE target districts.

Schools were divided into clusters (5-10 schools in each cluster) and lead data collectors were trained in all 4 districts. These lead data collectors reported the data of their clusters to the concerned ADOs/AEOs, who, after verification, submitted the data to senior district education officials and the District EMIS staff. RISE staff also assisted the education departments in conducting a seminar on findings and lessons learned on EMIS data for dissemination of education statistics in all 4 target-districts.

RISE officers found that there was not a very large pool of female managers and therefore the number of females trained under RISE Component 1 has been only 26. Initially RISE was not tasked to equally balance the gender of participants of training for education mangers but later, the project focus shifted to a greater gender balance in training especially for the training of teachers.

RISE was instrumental in assisting district governments with notifying and establishing district steering committees (SC) in all four districts. The mandate of the SC was to coordinate education activities, coordinate the various stakeholders’ activities and share issues/challenges and consider solutions for these problems.

This component utilized fewer funds (5% of total RISE budget expenditure) as compared with the other components.

Issues and Challenges

Transfers of Managers--Although RISE initially trained a discrete cohort of 161 education managers approximately 50% of these have left their respective offices across all four districts due to political interference, transfers and retirement. In Mansehra, senior officers indicated that they would try to encourage older members to train new in-coming managers who missed the initial training programs provided by RISE. The transfer of staff presents a challenge to the effective management of reforms that have been put in place by the project and also challenges the follow-through with using the monitoring tools that RISE has helped put in place as well as follow-through on budgeting and planning.

Sustainability of District Steering Committees—Notification has been granted for the formation and continuation of district steering committees. The sustainability of this forum is questionable once the RISE program support is terminated although RISE has documented steering committee meetings that have occurred since their support has ended. In Mansehra, three meetings have taken place without the support of RISE and there appears to be more enthusiasm for continuing these meetings as compared with AJK districts.

Implementation of District Education Plans and Budgets--Education managers in each of the four districts have been trained in developing need-based plans and budgets based on school monitoring data. In all target districts, RISE-embedded staff also assisted the education managers in developing budgets for fiscal year 2009-10 using EMIS and data collected during their school visits. There is no guarantee that the plans and budgets will be approved by the provincial and State governments as senior provincial and State officers have only had limited involvement in the planning and development phase; thus reducing the commitment and political leveraging required by provincial finance and planning and development offices for allocating funds against the RISE supported DEPs. District stakeholders in Poonch and Muzaffarabd Districts indicated that the Secretary of Education (AJK) had taken keen interest and even applauded the efforts of the planning and budgeting process at the district level. However, there has not been any indication that these plans and budgets were included in AJKs Federal Budget submission for 2010/11 Fiscal year.

Sustained EMIS reforms—RISE provided computers and workshops for EMIS staff on data entry, cleaning and generation of query-based reports from the Access Database. Moreover, 699 lead data collectors were also trained in the use of data collection tool and data collection procedures for Annual School Census (ASC) in AJ&K RISE target districts. While district education managers appreciated this support, they expressed their concern that there is no sanctioned post for a computer engineer and also there is no sustainable support office to which managers can seek help in computer issues. Also, it has been observed that there is no re-current budget for the maintenance of computers and other equipment that has been provided. In addition, data collection is one accomplishment, however the analysis and use of regularly collected data will require continuous support and training. Also, mobility support and conveyance allowances are not provided in the district budgets for the prompt collection of data.

Component 2: Teacher Training

RISE exceeded its goal of training 10,000 teachers by training 10, 316 teachers and a gender balance was achieved in the record that 5,346 males and 4,970 females have been trained. Teachers were selected jointly with education department and lists were coordinated with other donors. Schools were chosen based on CDM school lists initially and a needs assessment was conducted according to these lists. However, these lists kept changing so RISE developed its own consolidated list. After the initial school visits, a questionnaire was developed. RISE staff conducted a content analysis of the national curriculum that mapped all the themes in the curriculum. It was determined that there are two sets of curriculum: old one and the new scheme of studies. RISE chose to work with the old curriculum.

For determining the specific areas required for training, NEAS data were also taken into consideration according to grades 4 and 8 for student achievement in Math, Science and Urdu and that teachers required support in these content areas. The Quality Teacher Index was used to observe teaching skills (See Appendix F for TQI instrument). Observation played a significant role in tailoring the training programs. The baseline questionnaire was revised based on piloting.

In preparation of the training initiative, RISE selected and trained 361 master trainers from the government education system to conduct the trainings. Trainers were selected on the basis of their skills as instructors in consultation with the education departments in all four districts. Master trainers were selected from the ranks of government education institutions including pre-service and in-service training institutes, Directorates of Curriculum and Instruction, colleges, and higher secondary schools.

RISE also provided orientation training to master trainers that ranged from 6 to 10 days. They were first introduced to the concepts of active-learning methods, child-friendly behavior, and adult learning methods and then each presented lessons from the materials and received feedback from RISE staff and other participants. RISE provided continual support to the master trainers throughout their delivery of training to teachers through evaluation, feedback, and on-the-job support. The Table below shows the number of master trainers trained by district.

Master Trainers working with RISE

Source: RISE (April 2010), RISE Teacher Training Report, Islamabad

RISE’s one-year professional development program for teachers was a 3-step process intended to create sustainable change at the classroom level. The program began with a 12-day training during school vacations that focused on the use of active-learning techniques in English, mathematics, and science. At the end of this initial training, teachers were grouped into clusters of 7 to 15 from nearby schools. These clusters then met regularly over the course of the following year with a facilitator to review their progress, exchange model lessons, and plan active-learning lessons. The RISE professional development program ended with a 3-day follow-up training conducted a year after the initial training offering teachers a forum to discuss the changes that had taken place in their classrooms over the course of the year as well as things they learned from working with their clusters. The percentage of teachers who returned for the follow-up training was 90 percent. However, some teachers were unable to attend due to transfer to areas outside RISE working areas, retirement, or resignation.

Teachers Trained by District, Level, and Gender

Teachers

Source: RISE (April 2010), RISE Teacher Training Report, Islamabad

ADOs have prepared Professional Development Plans (PDP) up to 2012 by using the monitoring checklist. The PDP includes training plans developed for using the same master trainers to train the leftover teachers according to the RISE training design.

Monitoring and support—ADOs visited their schools and observing the classrooms and teachers and are giving constructive feedback. Regarding the question of which component was most effective in terms of supporting the education system, DEO replied that Components #1 and #3 were effective in reducing teacher absenteeism.

Gender Strategy

Ensuring equal access and participation in trainings for both men and women was a keystone of all of RISE trainings. RISE identified training venues that were close to the schools of participants to ensure easy access for all. Women especially were encouraged to attend the training through follow-up from RISE staff and education managers. RISE considered the cultural norms of each area and decided on different implementation strategies as needed. In Mansehra, all training classes were segregated by gender, while in AJK, decisions to arrange mixed gender training were made prior to the training in consultation with education managers and communities. If any participants in mixed-gender training felt uncomfortable, RISE was reportedly able to segregate the classes the same day and continue the training without interruption.

According to RISE officers, trainers were encouraged to ensure equal participation of men and women in discussions and group work. These strategies ensured that RISE met its goal of gender balance in all four districts. Overall, RISE trained 52 percent male and 48 percent female teachers. Poonch trained the highest percentage of female teachers with 51 percent, while Mansehra trained the lowest percentage with 46 percent. However, it should be noted that only 30 percent of all teachers in Mansehra were female.

Issues and Challenges

Selection of teachers—Education managers in Muzaffarabad and Mansehra Districts observed that the selection process for RISE teacher training programs was not carried out in full coordination with the education department although RISE officers indicate that lists of teachers were prepared and notified by the district education office. Perhaps the frequent transfer of officers has created the feeling for some of being left out of the teacher selection process.

District managers from Mansehra also observed that teachers selected for RISE training were not necessarily subject specialists who would benefit from specific math, science and English content training. RISE officers indicated however, that the training is based on activity-based techniques that could be applied across many content areas. Education managers and teachers also observed that neighboring schools and teachers should have been included in RISE training. The teachers not trained under the RISE initiative should be trained in the next phase.

Headteacher Training—Research conducted both globally and in Pakistan over the past two decades indicates that the headteacher is pivotal to the success of a public school’s performance . Stakeholders interviewed suggested that under RISE, the head teacher should have been included in initial project training. Although through the RISE Project, 405 headteachers were trained, stakeholder felt that the training of headteachers should have been provided at the beginning of the program rather than at the end. In years 2 and 3, RISE invited head teachers of middle and high schools to the last 2 days of the 12-day training. Primary schools do not have sanctioned head teacher posts, but instead have a teacher in charge. These teachers in charge participated in the entire 12-day training. During this time, head teachers observed active-learning methods and a model cluster meeting. This gave them exposure to the changes the teachers would be making in their classrooms. In year 4, RISE was able to add a 3-day training for head teachers. A majority of the teachers interviewed revealed that they need more support in the three subjects (Science, Math and English) and also in other subjects as well.

Establishment of teacher training capacity within existing institutions—Approximately 95% of in-service training for teachers in Pakistan is funded through external resources. RISE has funded and trained 10,316 teachers through a one-year cycle of support outside the government’s recurrent budget. While this training was required to help strengthen service delivery in schools affected by the earthquake, institutionalization of the training within existing training institutes did not occur under RISE. Project staff working under this component claimed that respective provincial and State governments had provided “No Objections” (NOC) that in fact promoted the widespread use of these training materials produced through RISE support. Unfortunately, these NOCs were sanctioned at the end of the RISE project and while RISE solicited support from subject specialists working within some of the teacher training institutes to develop training materials and to serve as master trainers, the institutions themselves were not trained/nor used to provide the project training nor was support provided to aid these institutes in their annual planning and budgeting for in-service teacher training.

The selection and training of master trainers from within district pools of subject specialists and trainers has led to a trained cadre of professionals; however, there is no recurrent budget for their continued service and there is no guarantee that government institutions will be able to carry forward the training to the teachers who were not trained in the RISE target districts.

The Directorate for Teacher Education (DCTE) in KPK or the Directorate of Education Extension (DEE) in AJK receives very little State funds for annual in-service programs. For example, AJK receives only Rs.180, 000 ($2,118) annually to provide in-service training for a total teacher workforce of 30,000 across 10 districts. The RISE project support targeted three of these 10 districts. Stakeholders were unanimous that the RISE training should be further delivered to all teachers after the project ends. However there is no recurrent budget to accommodate the RISE training nor any other training if it is to be provided through government development budgets. New programs should work more closely with existing teacher-training institutes to leverage State funds in order to institutionalize the actual delivery of in-service training that is developed with them e.g. DCTE, DEE, Government Colleges, etc. Also, teacher-training officers in AJK indicated that in the future, programs should agree on a clear set of TORs with government departments such as DCTE, DCRD and DEE. Under RISE, it was highlighted that there were no direct TORs agreed between the project and these government institutions. It was suggested that appraisal of any new training programs should be participatory involving more members of these training institutes and based on local demands.

Timely delivery of kits and materials—Teachers and Master Trainers indicated that teaching kits and other material should have provided along with the training rather than at the end of the project. RISE staff explained that due to cost savings related to the favorable Rupee/Dollar exchange rate, the Project was able to procure further resources for the target schools at the end of the project although some sets of materials were previously delivered to target schools.

TA/DA and other training incentives—A majority of stakeholders interviewed indicated that RISE should have enforced a flat rate for TA/DA to teachers who participated in various project training. Government TA/DA rates were alleged to be outdated and not in line with current inflation. RISE officers claimed that there was no way to verify mileage and other details of expenditures claimed by training participants and therefore a flat rate would help to equalize the training incentive. Also, officers indicated that too much time/resources were expended on lunch. It was suggested that lunch should not be provided in future training as it became too costly and difficult to arrange under the RISE Project.

Sustainability of Learning Resource Centers (LRC), Professional Development Forums (PDF) and clusters—RISE was instrumental in fostering continual support for teachers through different fora including LRCs, PDFs and cluster meetings annually throughout the program. These mentoring and cluster support mechanisms while important are unfortunately not sustainable unless existing teacher training institutes actually own the processes and have a stake in planning activities and are able to allocate even limited resources for their continuation. Resource centers in Pakistan have a “checkered” past with the success heavily dependent on motivated school heads or strong district officer support. Officers interviewed indicated that LRCs, PDFs and clusters would most likely cease after RISE support ended. Some officers indicated that LRCs should have been established within the Government Colleges to promote lasting coordination and operation with the government departments.

Institutionalization of research and continual assessment—Officers within teacher training institutes in the RISE target areas indicated that the research that RISE conducted on its project efforts should have been conducted with full participation of their staff. For example, in AJK there are two agencies—Directorate for Curriculum, Research and Development (DCRD) and Kashmir Education Assessment System (KEAS). Officers within these institutions indicated that they could have benefitted from RISE research if the research, in fact, had been designed and coordinated with their participation. They further added that this embedded support would have promoted sustainability of the research functions once the RISE Project ended. DCRD, KEAS nor DEE have ever conducted, for instance, a comprehensive baseline of in-service training across all 10 districts of the State of AJK. Although RISE was concerned with only 3 districts, the project could have seized the opportunity to train these institutions to broaden the research agenda from 3 districts to 10 to gain a better understanding of the needs of its teachers in order to plan better for annual in-service delivery.

Component #3: Community Participation in School Management

The aim of this component was to increase community participation in school management and school improvement. RISE’s community mobilization approach involved formation/revitalization and strengthening of SMCs/PTCs through short-term training, sustained on-the-job technical support, and activities for SMCs/PTCs orchestrated to promote peer learning. The capacity of the SMCs/PTCs to plan and manage was further strengthened through small grants awards in which the SMCs/PTCs undertook projects that they had designed. RISE revitalized 2,300 SMCs/PTCs in all four districts and trained over 16,500 SMC/PTC members and 2,049 women as honorary SMC members in AJK thus achieving all of its planned targets.

In addition, RISE aimed to strengthen SMCs/PTCs as institutions. Strengthening institutions was defined as: 1) enabling SMC/PTC members to understand their roles and responsibilities as well as the roles and responsibilities of teachers and education officials in promoting quality education and 2) enhancing the SMCs/PTCs abilities to function effectively. The objective was to help SMCs/PTCs become self-sustaining entities. RISE associated partners such as SRSP, Sungi, NRSP and IRC to work with this component and collectively, approached this component with open participation in an “all partners” workshop. Partners were engaged in a review of their collective databases to review their community mobilization experiences and materials. After this review, the RISE approach was developed in which an 18-month cycle of support for each SMC/PTC was prepared as a roadmap for implementing partners.

The first six-month was considered a mobilization period followed by 12 months of ongoing support. The initial mapping of SMC/PTCs highlighted that other organizations had been working with these communities but mostly on a short-term basis. It was apparent that the committees needed to be reactivated. Training was designed to emphasize participation management and to strengthen an understanding of roles and responsibilities and how to manage conflict/conflict resolution with the government education departments and teachers. Ten topics were identified for training including introduction to school improvement planning. At first, communities highlighted school infrastructure as their primary roadblock to improved education but with time and training, communities were able to focus on soft component priorities for improving their schools. Also 50% of SMC/PTCs were provided small grants while the remaining were provided support and encouragement for applying for grants from either government or other donor sources.

See the diagram below illustrating the process that RISE adopted for supporting SMC/PTCs in the target areas.

RISE Community Support Process

Source: SRSP powerpoint presentation (2010), Mansehra.

Issues and Challenges

RISE small grant process—RISE implementing partners unanimously indicated that the RISE grants process was very cumbersome and time consuming and that it should be streamlined if implemented in the future. The claim was made that the forms were in English, at first, and were difficult to understand for communities. Also it was highlighted that there were too many approval layers. First the RISE district office was expected to review the application and revise if necessary. Then the forms are sent to Islamabad for review and then they are submitted to Washington, DC for final approval. None of the stakeholders interviewed could explain why the forms had to be submitted to Washington AIR headquarter office for final approval. Officers also indicated that the grants were provided in two installments whereas there should have been only one installment to avoid another lengthy processing period. The suggestion was made that the approval process should have been decentralized to the district level for grants.

Project officers and stakeholders all indicated that at first, RISE had introduced a policy whereby all targeted PTC/SMCs were to receive a RISE small grant of Rs. 45,000. After the policy had been introduced and announced, RISE management changed the policy and only 50% of the targeted committees actually received the RISE grants due to project budget constraints. This reportedly caused mistrust between communities and implementing partners particularly with the communities that did not receive the RISE grants.

Institutional fatigue, donor dependency and sustainability—When queried about the sustainability of SMC/PTCs, implementing partners and particularly NRSP indicated that the exit strategy employed under RISE was to link the school committees with other rural development committees and organizations for continued support. In AJK, for instance, RSPN indicated that SMCs were being linked with Local Support Organizations (LSO) in order to make applications for funds that would accumulate through the support of NRSP and other donors.