Attachment_No._6_-_TCPP_QR_July-Sept_2016_Public_Version.pdf

EducAcción Project

Quarterly Progress Report

July 1, 2016– September 30, 2016

Submitted by:

American Institutes for Research

With:

CARE Honduras Catholic Relief Services Honduras ChildFund Honduras Fundación Aned Fundación para la Educación Ricardo Ernesto Maduro Andreu (FEREMA) Save the Children Honduras

U.S. Agency for International Development Cooperative Agreement No. AID-522-A-11-00003

SUMMARY OF THE PROJECT

Title of Project: Teacher-Citizen Participation Project (Proyecto EducAcción)

Prime Contractor: American Institutes for Research (AIR)

Cooperative Agreement No.: AID-522-A-11-00003

Reporting Period: April 1, 2016-June 30, 2016

Budget: $xxx

Start Date: July 7, 2011

End Date: February 28, 2017

Contact For the Contractor: xxx, Principal Project Specialist AIR, 1000 Thomas Jefferson St. NW Washington, DC 20007 Tel: 202 403-5261 Fax: 202 403-5979 Email: xxx mailto:bdrury@air.org

Contents I. Introduction II. Activities Implemented During the Quarter III. Results IV. Number of Participants Trained V. Activities Planned for Next Quarter VI. Lessons Learned and Recommendations

I. Introduction

This report documents the activities of the USAID-funded Honduras TCPP (EducAcción) Project during the third quarter of the 2016 calendar year, from July 1st to September 30th.

The project is implemented by the American Institutes for Research (AIR), with partners CARE, Catholic Relief Services (CRS), ChildFund, Save the Children, Fundación para la Educación Ricardo Ernesto Maduro Andreu (FEREMA), and Fundación Aned.

This Quarterly Report is organized into the following sections: 1) Activities Implemented During the Quarter, 2) Results, 3) Number of Participants Trained, 4) Activities Planned for Next Quarter, and 5) Lessons Learned and Recommendations. Each section is differentiated by result. Additionally, the annexes include 1) Summary of Training Events,

2) Summary of Issues Encountered by Result, 3) Annual Indicator Summary, 4) and Summary Performance Data Table.

In May 2015, the USAID Mission in Honduras awarded the TCPP Project additional funding to implement a new component as part of a rigorous impact evaluation, which is being carried out by Mathematica. This evaluation studies the application and use of formative assessments and end of grade tests to determine their impact on students’ academic achievement and early reading skills from 1st to 3rd grade. The TCPP Project team implements the intervention, in accordance with Mathematica’s impact evaluation design. The intervention has been incorporated in the project as Result 3, the Promising Reading Interventions (PRI) Component.

II. Activities Implemented During the Quarter

2.1. Technical Implementation Activities

This section summarizes the technical activities implemented in order to achieve project results.

2.1.1. Result 1: Continued support to reach EFA goals

Activities summary and achievements

Almost all Result 1 activities were finished by July 2016. The only remaining activities during this period involved data on project activities in the field collected by the project’s monitoring and evaluation (M&E) team during the previous quarter. During this quarter, the M&E team processed the data for annual data reporting. The data will be used to verify project results and to produce a preliminary achievement report.

2.1.2. Result 2: Strengthened community participation in education

Activities Summary and Achievements

As with Result 1, activities under Result 2 were in almost all cases completed previous to this quarter. There was very limited remaining work this past quarter due to the processing of data collected by the M&E team for indicators related to Result 2.

Additionally, during this quarter, the team continued and finalized work related to the school libraries. The EducAcción team received 14,781 books from the International Book Bank and printed another 9,500 copies of the story anthologies produced under the project. Using these books, project personnel prepared and delivered libraries for 69 communities or schools. To ensure that the libraries are used for improving children’s reading skills, project staff provided technical assistance and guidance to 235 teachers from the schools receiving libraries. The teachers will be responsible for promoting creative techniques to encourage reading and to develop a plan to promote reading at school.

2.1.3. Result 3: Promising Reading Interventions

Activities Summary and Achievements

During this quarter, TCPP staff implemented the technical activities planned for this period. These activities are presented by school group as follows:

Groups A and B

• Provide guidance and monitor the implementation of the school plan, emphasizing the improvement of students’ reading skills. To achieve this activity, the following sub-activities must be implemented: provide advice on how to revise and validate an action plan, monitor the implementation of the action plan through follow-up visits with the principal and school personnel, and provide principals with feedback on school plan implementation. The levels of implementation of these sub-activities are presented in the next table.

Table 1: Level of implementation of the sub-activities related to the activity

Sub-activity Amount of school by level of implementation

Group A Group B

Completed Not completed Completed Not completed Monitor the implementation of the school plan through follow-up visits with the principal and school personnel

60 0 61 0

Provide feedback to the school principal on the fulfillment of the school plan 60 0 61 0

After the school plan was developed, each pedagogical advisor met with the school principal to evaluate the activities included in the plan using the following criteria: viability, verifiability, pertinence, coherence, and measurability. This review usually resulted in eliminating or adjusting some of the school plan activities that were initially proposed.

To date, all schools in Group A and B have received one consultative visit and one follow-up visit from the pedagogical advisor to monitor the implementation of the school plan and to provide feedback on the plan’s progress. The implementation of these sub-activities depends on how much time was needed to develop and validate the school plan and how many schools have been assigned to each pedagogical advisor.

Both of the aforementioned sub-activities are carried out in collaboration with the school authorities. To monitor plan implementation, the school principal self-evaluates his or her school implementation plan. The principal reviews each activity and determines implementation progress and challenges. Based on the evaluation results, the principal agrees to address or improve the status of the implementation plan as needed.

The table below shares the main findings of pedagogical advisors’ follow-up to the initial school improvement plan meetings. The commitments assumed by principals and teachers as a result of this are as follows:

Factors to Improve Pending Activities for Teachers, Administrators, and Pedagogical Advisors

Lesson planning

1. Teacher plans based on pacing guides: In most Group B schools, teacher plans are not designed to address and improve standards where students demonstrate low performance.

2. The remedial plan strategies for Group A schools need to be revised.

1. Principals will check and oversee teacher plans with a comprehensive strategy to improve student performance in reading comprehension.

2. Pedagogical advisors will continue to work with teachers on remedial plans with a focus on the use of tools and strategies to improve standards with low performance.

These strategies and tools were part of the last training workshop.

Supervision

3. The pedagogical advisors observed improved supervision. Nonetheless, principals need to provide feedback to teachers based on findings about their performance in the class.

1. Principals will monitor and supervise teaching practices in the classroom during the last school quarter.

2. In most cases, principals will have meetings with supervised teachers to provide feedback.

Student performance

4. School principals are not analyzing reading comprehension test results for first cycle students.

5. School principals are not monitoring the implementation of reading comprehension strategies in the classroom.

1. Principals must analyze reading comprehension evaluation results.

2. Pedagogical advisors have recommended that principals oversee the implementation of strategies to improve reading comprehension.

3. Pedagogical advisors will work with principals to monitor the school improvement plan in student performance areas.

School Environment

6. There is little and inconsistent supervision from principals in Group B schools regarding the use of materials in the learning centers.

7. Few activities in the school action plan have been designed to improve student reading comprehension.

1. Principals were invited to use school civic holidays and activities as opportunities to engage students with low reading performance. The school can provide students with reading material based on holidays and civic festivities, which will provide them with an opportunity to improve their reading skills.

Community Participation and Communications

8. The sharing of evaluation monthly results with parents should improve in Group A schools. In Group B schools, summative evaluation results should also be shared and discussed with parents every quarter.

9. The parents’ meetings should incorporate the analysis of the student performance and ways to improve reading comprehension, and homework review.

1. Pedagogical advisors will continue working with principals on their role in the activities that directly impact students’ performance and involve other stakeholders.

2. Principals will be proactive in analyzing the previous quarter’s test results with students’ parents.

Group A

Per the project’s work plan, the following activities and sub-activities were implemented only in Group A schools.

• Training in the administration, scoring and analysis of the monthly formative test results, including results-based decision-making that result in remedial plans and/or adjustments to teacher planning. The status of the sub-activities related to this activity is presented in the next table.

Table 2: Level of implementation of the sub-activities related to the activity

Sub-activity Number of schools by level of implementation Completed Not completed

Provide the principal and teachers with technical advice on the analysis and use of formative assessments, reprogramming, adjustment or modification of teacher class plans, as appropriate

60 0

This sub-activity was implemented in the 60 schools at least once per month during this quarter. TCPP staff provided principals and teachers with technical guidance, which resulted in teachers using formative test results to prepare a remedial plan by grade and section. The remedial plan included content standards with low performance, activities to improve student performance in those standards, resources needed for the activities, and a timeline for implementing the activity.

The process of analysis and decision-making has helped teachers and principals in Group A schools use formative tests to improve student reading skills using Spanish language test items linked to reading comprehension.

• Monthly technical monitoring on administering the formative assessment, tabulating test results, and implementing remedial plans or adjusting teacher classroom plans. The status of the sub-activities related to this activity is presented in the next table.

Table 3: Level of implementation of the sub-activities related to the activity

Sub-activity Amount of school by level of implementation Completed Not completed

Schedule visits with teachers to support their use of the formative tests 60 0

Provide the teachers with technical advice on adjusting and implementing teacher classroom plans or remedial plans

60 0

Both sub-activities were implemented during this quarter. The visit to provide support was first scheduled with the school principal and the teachers. The schedule includes at least one visit per month during and after the application of the formative test in the classroom.

During the scheduled visits, the pedagogical advisor provides teachers with guidance in the administration of formative assessments, tabulating test results, and analyzing the results. The pedagogical advisor encourages the teacher to reflect on the use of the test results and self-evaluate the quality of the activities included in their plans. The criteria to evaluate the quality of the remedial classroom teaching plans are the same criteria described previously to evaluate the activities included in the school plan.

As a result of the consultation visits, the pedagogical advisors determined that the teachers are not spending much time developing the school improvement plans.

Moreover, some teachers still use traditional instructional strategies to strengthen pedagogical content. In extreme cases, some teachers continue to use a standard teacher planning document obtained from another colleague.

After teacher training during the previous quarter, the pedagogical advisors developed a series of tools with pedagogical strategies to improve technical assistance. By using these tools, they have successfully encouraged teachers to use more appropriate strategies to develop better school improvement plans. According to some school authorities, some first cycle students’ monthly progress in Spanish language academic performance has improved in line with the teacher pedagogic support that has been carried out.

Intervention Fidelity The project team reports on the level of adherence to the intervention model’s key components each quarter. Intervention fidelity is measured using the information that the pedagogical advisors provide in their monthly reports. Their monthly reports indicate the type and amount of sub-activities that they implemented during the quarter. This information is then compared to the sub-activities planned in the PRI component’s annual work plan.

The sub-activity goals were met based on the adherence level for each school and school group. Below is the scale used to determine the level of adherence by school and school group.

Table 5: Adherence Level by School and School Group

Adherence level by school Adherence level by school group Level of adherence Criteria Level of adherence Criteria

Low Less than 69% of the activities planned have been implemented in the school

Low Less than 69% of the schools in the group achieved high fidelity

Intermediate 70% to 89% of the activities planned have been implemented in the school

Intermediate 70% to 89% of the schools in the group achieved high fidelity

High 90% to 100% of the activities planned have been implemented in the school

High 90% to 100% of the schools in the group achieved high fidelity

The next table shows the number of schools by adherence level and school group.

Table 6: Amount of School by Adherence Level

Group Adherence level Total schools served High Intermediate Low A 60 0 0 60 B 60 0 0 61

Table 7: Adherence Level by Group

Group Percentage of schools with high adherence

Level of adherence by school group

A 100% High B 100% High

Challenges and Perspectives

During the past two quarters (including the one currently being reported on), the pedagogical advisors have implemented the original workplan with full fidelity. Their actions are influenced by the project management, which focuses on compliance with the planned activities on one hand, and consistent school activities that correspond to the academic calendar on the other hand. The exceptions to consistent school activities include the July celebrations in the department of Lempira, the patriotic festivals in the first and second weeks of September, and the departmental faculty meeting that took place Santa Barbara.

III. Results

This section includes the annual status of the indicators defined in the PMP, which were updated in 2014, and are reported annually in the third quarter of each calendar year. The Annual Indicator Summary can be found in Annex 3 and Annex 4 of this report

IV. Number of Participants Trained

During this quarter, no additional participants were trained.

V. Activities Planned for Next Quarter

The project staff will implement the following activities next quarter:

Results 1 and 2:

There are no upcoming planned activities for these two results, which are now complete.

Result 3: Result 3 activities that will be carried out in the upcoming quarter include the below:

o Provide technical assistance and monitor each school’s implementation of the school action plan. The school action plan emphasizes improved reading skills among first cycle students.

o Provide training in the administration, scoring and analysis of the monthly formative test results. Training includes results-based decision-making that result in remedial plans and/or adjustment to teacher planning.

o Technical monitoring of application and scoring of monthly formative tests and monitoring of remedial plans and/or adjustment to teacher planning.

o Development of an analytical report on schools’ achievements in the activities supported by the Project over the past year. This report will be shared and discussed early next year.

o Support the end of grade test application in the 180 schools that comprise the PRI study.

o Train all of the first cycle teachers in Group B and C schools in Tegucigalpa and La Ceiba. The training will provide teachers with the skills to use formative assessments and prepare remedial plans based on test results. Based on departmental authorities’ decision, and given the availability of formative assessments in Santa Barbara and Lempira, the team will train all first and second cycle teachers in both departments. The team will use teachers from Group A schools to facilitate the training because of their recent experience and willingness to share their experience with other teachers.

o Print and distribute formative assessments to the 180 Promising Reading Intervention (PRI) schools to be used in the 2017 academic year. The team will also prepare instructional and educational programming materials for teachers’ use.

VI. Lessons Learned and Recommendations

The educational authorities at the municipal, district, and departmental levels provide significant support to the formative assessments application process. They have identified the tests as a tool to improve student academic performance in their respective fields of management.

Since the previous quarter, the EducAcción team witnessed the professional development of certain teachers in Group A schools who are now resources in their locations and are training their colleagues. These teachers are now in demand to act as formative assessment trainers within educational networks in Lempira and Santa Barbara. If they continue to act as trainers in this capacity, these teachers may eventually provide leadership in disseminating and using formative assessments and their results, and in developing remedial plans to improve student reading skills.

Annexes

Annex 1: Summary of Training Events during the July 2016-September 2016 quarter Annex 2: Summary of Issues Encountered by Result Annex 3: Annual Indicator Summary Annex 4: Summary Performance Data Table