J.6_EGRA_C1_FU2_Briefer_12Dec_2014_USAID.pdf

December, 2014

USAID/Uganda School Health and Reading Program: Cluster 1 EGRA Follow-Up 2

Has reading achievement increased as a result of the USAID/Uganda School Health and Reading Program interventions? Early Grade Reading Assessment data collected for 4 Cluster 1 languages (Ateso, Leblango, Luganda and Runyunkore-Rukiga)1 and English at baseline (beginning of P1 February, 2013) compared to data collected at the end of P2 (October, 2014) show increases in fundamental reading skills, significantly higher than increases found in control schools.

By the end of P2, in the local language, learners in Program Schools could read more words of a reading passage and answer more questions about that passage than learners in control schools – this was true in all 4 Local Languages. Learners in program schools were at least two times more likely to read 20 or more words per minute in all 4 local languages then learners in control schools. Learners could also read more words in English.

These findings provide substantive evidence that the USAID/Uganda School Health and Reading Program interventions are having a positive impact on reading achievement in program schools.

The first follow up, conducted at the end of P1 in October, 2013, showed very modest gains in pre-reading tasks in all language groups not significantly higher than control gains (except in the case of Luganda, where significant gains above control schools were found). This was not unexpected as the pupil primers --a central feature of the intervention-- did not reach the learners until the end of the school year (in many cases, after follow-up data collection) due to initial program implementation delays. These follow up findings suggest that P2 interventions have boosted learners achievement even in the absence of exposure to the full program in P1 leading to significant improvements in beginning and transitional reading skills over and above those found in control schools.

In general, there was little significant difference between girls and boys – in those instances where significant differences were found, it was almost always the case of the girls outperforming the boys.

1 These schools started in the program in January, 2013 at the beginning of P1. Learners in program (treatment) and control schools were assessed at the beginning of P1, the end of P1 and the end of P2.

MINISTRY OF EDUCATION AND SPORTS December 12, 2014

Oral Passage Reading and Comprehension: Improvements in Oral Reading Fluency (ORF or words per minute) in the local language were notable from baseline to this follow-up and were significantly greater in treatment schools and control schools.

Figure 1 shows the percent of learners who could read at least 20 words per minute at the end of P2. At baseline, all language groups (treatment and control) started from a baseline of zero (with the exception of Runyunkore-Rukiga where the control baseline percent was 1). In Ateso, the percent of learners reading 20 or more words a minute remained at 0 for learners in control schools, in program schools it increased to 2.8%. In Leblango, 4% of program learners could read 20+ words per minute, compared to 1.5% for control. In Luganda, 9% increase in control schools compares to 18% for program schools and for Runyunkore-Rukiga, the percent of learners reading 20 or more words per minute increased to 16.4% compared to an increase of only 5.5% in control schools2. In all 4 language groups, P2 learners in program schools were between 2 to 3 times more likely to read 20+ words per minute in the local language than P2 learners in control schools.

Figure 2 shows the increase in words read per minute or oral reading fluency. Where P1 learners could read no words in one minute in their local language at baseline, they can now read between 2 and 10 words per minute in program schools compared to less than 1 to 5 words per minute in control schools. This is still below any benchmark for reading (such as one cited of 30 wpm for end of P2) though caution needs to be taken when comparing languages such as those found in Uganda which have a much longer word length 3.

2 All of the differences between program and control schools are statistically significant.

3 See Abadzi, 2012 “Developing cross-language metrics for reading fluency measurement”. Abadzi provides an example of increasing reading fluency measures by 30% to account for the difference in word length in other Bantu languages.

USAID/Uganda School Health and Reading Program EGRA Cluster 2 Follow Up 2 Results: December, 2014

0 1.9 0.1 2 0.2

9.2

0.3

6.9

0 0.5 0 0.9 0.1

5.5

0.3

4.4

Baseline FU2 Baseline FU2 Baseline FU2 Baseline FU2

Ateso Leblango Luganda Run/Rukiga

Figure 2: Local Language Oral Reading Fluency ( Words Per minute) Baseline (March, 2013) and Follow Up 2 (October, 2014)

School Health and Reading Program

Control

Learners need to read with fluency but they also need to be able to understand what they are reading. Figure 3 shows the percent of the local language reading passage questions that learners were able to answer correctly at end of P2. At baseline (at the beginning of P1), no learners could answer a single question in either program or control schools. At the end of P2, that percent had increased to between 20 and 60% in program schools: 20% in Ateso and Leblango (compared to 0 for control schools) and 60% in Luganda and Runyunkore-Rukiga (compared to 30% and 40% in control schools)4.

20 20

60 60

0 0

Ateso Leblango Luganda Run/Rukiga

Figure 3: % of Local Language Reading Passage Questions Answered Follow Up 2 (end of P2)

School Health and Reading Program

Control

English Results: As expected, most of the significant differences between treatment and control schools at follow up were in local language reading skills as the program teaches reading in the local language following the MoES thematic curriculum which uses the local language as the medium of instruction in P1 to P3. That being said, there were significant differences in English reading achievement as well as shown in Figure 4.

4 All differences are statistically significant.

1 12 1 16 13

1 1 0 10 6

Baseline FU 2 Baseline FU 2 Baseline FU 2 Baseline FU 2

Ateso Leblango Luganda Run/Rukiga

Figure 4: Percent of Learners Who Could Read At Least One Word of an ENGLISH Story

Baseline and Follow-Up 2

School Health and Reading Program Control

At the end of P2 follow-up, 12% of learners in Ateso speaking program schools could read at least one word of an English story. This is up from only 1% at baseline and compares to NO learners at follow up in control schools. In Leblango, the percentage increase from 1 at baseline to 16% at follow up (compared to 10% in control schools). For Luganda the percentages of learners at follow up reading at least one word of an English story increases from 13% to 50% in program schools and from 6% to 35% in control schools. In Runkore- Rukiga the percent increase from 4 to 51% in program schools and from 6% to 42% in control schools.

There were significant differences in the percent of learners reading 20 or more words a minute in English as well as shown in Figure 5. While all groups started at zero at baseline, Ateso and Leblango have improved modestly at follow up to 1.3% of learners reading 20+ WPM compared to 0 for control schools. Almost 25% of learners in Luganda speaking program schools could read 20+ WPM compared to only 14.3% in control schools. In Runyunkore-Rukiga speakings schools, the gains were similar in program and control schools with 8-9% of learners reading 20 or more words per minute.