Ref 7. Tajikistan RWM- Final EGRA Baseline Report.docx
DOCX document 1 MB Posted
- Attached to
- Learn Together Activity in Tajikistan
- Solicitation number
- 72011520R00009
View the file
Other files
Show all 28
On GovTribe
Work with this file
- Download the original file
- Contacts named in this file
- Similar government files
- Ask GovTribe AI about this file
Text version
Early Grade Reading Assessment, Baseline Report
USAID READ WITH ME PROJECT
Photo credit: USAID Read with Me
July 30, 2018
This publication was produced for review by the United States Agency for International Development. It was prepared by School-to-School International and Chemonics International Inc
Early Grade Reading Assessment, Baseline Report
USAID READ WITH ME PROJECT
Contract No. AID-176-C-16-00003
DISCLAIMER
The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States government.
Acknowledgments Many organizations and individuals contributed to the successful completion of the Read with Me Project (RWM) Baseline Early Grade Reading Assessment (EGRA). The Ministry of Education and Science (MoES), National Testing Center, Academy of Education, Education Development Institute under the Academy of Education, Republican Methodological Center, Republic Training and Retraining of Education Specialists Institute with its five regional teacher training institutes and the district or city education departments provided insight, expertise and guidance throughout the assessment process from the review of assessment tools to data collection monitoring. We would like to extend special gratitude to Mr. Nuriddin Said, Minister of Education and Science, and Ms. Latofat Naziri, Deputy Minister of Education and Science, for their guidance and support, as well as to Mr. Abdujabbor Aliev, RWM Coordinator from MoES, for his assistance throughout the assessment.
This report would not have been possible without the support of the American people through the United States Agency for International Development (USAID) RWM project. USAID provided critical input on the EGRA process through Dr. Amy Scott, USAID Central Asia Education Development Advisor; Ms. Katherine Crawford, USAID Tajikistan Country Office Director; and Mrs. Mavjuda Nabieva, USAID Tajikistan Contracting Officer’s Representative (COR).
The EGRA was implemented with the technical assistance of School-to-School International (STS) and Chemonics International. The assessment was planned and administered with the support of staff members in the United States and in the USAID RWM project office in Tajikistan. The authors appreciate RWM project staff’s collaboration and support. We also appreciate the hard work and many other individuals at STS, including Elizabeth Odenwald, Drew Schmenner, and Melyssa Sibal for contributions to tool development, programming, data collection training, data analysis, and producing this EGRA Baseline report. We thank Laura Zasoski for her editing.
We thank M-Vector Research for providing organization and logistical support during the EGRA baseline data collection, and the 68 data collectors, whose efforts to reach schools in every region of Tajikistan made this report possible. Finally, we thank the school directors, teachers, and students who welcomed data collectors to their schools and participated in this study.
The following persons played a major role in the process:
| Terry Giles | (Chemonics International) | |
| Hurmat Dushanbiev | (Chemonics International) | |
| Valerie Power | (Chemonics International) |
| Peter Cooper | (School-to-School International) |
| Kristina Solum | (School-to-School International) |
| Adiba Kosimova | (School-to-School International) |
| Matthew Murray | (School-to-School International) |
| Autumn Thomas | (School-to-School International) |
| Louise Bahry | (School-to-School International) |
| Allison Born | (School-to-School International) |
| Fernanda Gandara | (School-to-School International) |
| Sydney Merz | (School-to-School International) |
| Randy Tarnowski | (School-to-School International) |
| Hetal Thukral | (School-to-School International) |
Contents
| Acknowledgments | 4 |
| Contents | 5 |
| Acronyms | 7 |
| List of tables | 8 |
| List of figures | 11 |
| Executive Summary | 13 |
| Introduction | 21 |
| RWM project background | 21 |
| Early grade reading outcomes in Tajikistan | 22 |
| Purpose of the study | 23 |
| Research questions | 23 |
| Methods and Limitations | 25 |
| Tool development | 25 |
| Baseline data collection sampling | 26 |
| Data collection | 28 |
| Analytic methods | 29 |
| Limitations | 31 |
| EGRA results | 32 |
| Part 1: National-level EGRA results across years | 32 |
| Part 2: 2018 EGRA results by gender | 34 |
| Part 3: 2018 EGRA results by location | 35 |
| Part 4: 2018 EGRA results by region | 37 |
| Part 5: 2018 EGRA results by school group | 40 |
| Part 6: Achievement of grade-level reading benchmarks | 44 |
| Contextualizing EGRA results | 46 |
| Student questionnaire results | 46 |
| Teacher and school director questionnaire | 50 |
| School and classroom inventory | 52 |
| Predictors of reading outcomes | 53 |
| Summary of findings and discussion | 56 |
| Summary of findings | 56 |
| Discussion | 57 |
| Recommendations | 58 |
| Bibliography | 59 |
| Annexes | 60 |
| Annex A: EGRA adaptation and piloting | 61 |
| Annex B: Operational data collection | 66 |
| Annex C: Analytic methods | 69 |
| Annex D: Concept note | 71 |
| Annex E: Reliability of the EGRA subtask items | 77 |
| Annex F: Intraclass correlations | 81 |
| Annex G: Distributions of oral reading fluency scores | 82 |
| Annex H: EGRA results figures | 84 |
| Annex I: EGRA tables and results | 91 |
| Annex J: Detailed student survey tables | 108 |
| Annex K: Detailed SSME tables | 117 |
Acronyms
| EGRA |
| Early Grade Reading Assessment |
| CWPM |
| Correct Words per Minute |
| DRS |
| Districts of Republican Subordination |
| EMIS |
| Education Management Information System |
| GBAO |
| Gorno-Badakhshan Autonomous Oblast |
| IRB |
| Institutional Review Board |
| IRR |
| Inter-rater reliability |
| LOI |
| Language of Instruction |
| MoES |
| Ministry of Education and Science |
| ODK |
| Open Data Kit |
| ORF |
| Oral Reading Fluency |
| QCO |
| Quality Control Officer |
| QLP |
| Quality Learning Project |
| QRP |
| Quality Reading Project |
| RWM |
| Read with Me |
| SD |
| Standard Deviation |
| SE |
| Standard Error of the Mean |
| SES |
| Socioeconomic Status |
| SSME |
| Snapshot of School Management Effectiveness |
| STS |
| School-to-School International |
USAID
WG
United States Agency for International Development Working Group
List of tables
| Table 1: Baseline EGRA Subtasks | 26 |
| Table 2: Baseline SSME Tools | 26 |
| Table 3: Number of Students and Schools Required by Group | 27 |
| Table 4: Number of Students and Schools Required for National Results | 27 |
| Table 5: Number of Students and Schools Sampled | 27 |
| Table 6: Timeline of RWM Interventions by Cohort | 29 |
| Table 7: Trends in Grade 2 and Grade 4 Student Reading Performance in Tajik by Subtask and Year | 33 |
| Table 8: Trends in Grade 2 and Grade 4 Student Reading Performance in Russian by Subtask and Year | 33 |
| Table 9: Trends in Grade 2 and Grade 4 Student Reading Performance in Tajik by Subtask and Gender, 2018 | 34 |
| Table 10: Trends in Grade 2 and Grade 4 Student Reading Performance in Russian by Subtask and Gender, 2018 | 35 |
| Table 11: Trends in Grade 2 and Grade 4 Student Reading Performance in Tajik by Subtask and Location, 2018 | 36 |
| Table 12: Trends in Grade 2 and Grade 4 Student Reading Performance in Russian by Subtask and Location, 2018 | 36 |
| Table 13: Tajik Grade 2 EGRA Rank-Order Mean Scores by Region across Subtasks, 2018 | 37 |
| Table 14: Tajik Grade 4 EGRA Rank-Order Mean Scores by Region across Subtasks, 2018 | 38 |
| Table 15: Russian Grade 2 EGRA Rank-Order Mean Scores by Region across Subtasks, 2018 | 39 |
| Table 16: Russian Grade 4 EGRA Rank-Order Mean Scores by Region across Subtasks, 2018 | 39 |
| Table 17: Correlations between Student Survey Items with Variance and Oral Reading Fluency by Grade and Language | 53 |
| Table 18: Correlations between SSME Items with Variance and Oral Reading Fluency by Grade and Language | 54 |
| Table 19: EGRA Subtasks and Skills | 61 |
| Table 20: SSME tools | 63 |
| Table 21: Form construction | 64 |
| Table 22: Summary of completed Grade 2 EGRA by language and form | 65 |
| Table 23: Summary of completed Grade 4 EGRA by language and form | 65 |
| Table 24: Number of Enumerators | 66 |
| Table 25: Number of Teams and Dates of EGRA Baseline Administration | 67 |
| Table 26: Equating Methodologies | 69 |
| Table 27: Correlations between EGRA Subtasks, Tajik Grade 2 | 77 |
| Table 28: Correlations between EGRA Subtasks, Tajik Grade 4 | 77 |
| Table 29: Correlations between EGRA Subtasks, Russian Grade 2 | 77 |
| Table 30: Correlations between EGRA Subtasks, Russian Grade 4 | 78 |
| Table 31: 2018 EGRA Reliability Estimates, Tajik Grade 2 | 78 |
| Table 32: 2018 EGRA Reliability Estimates, Tajik Grade 4 | 78 |
| Table 33: 2018 EGRA Reliability Estimates, Russian Grade 2 | 79 |
| Table 34: 2018 EGRA Reliability Estimates, Russian Grade 4 | 79 |
| Table 35: Inter-Rater Reliability Estimates by Subtask, Tajik | 80 |
| Table 36: Inter-Rater Reliability Estimates by Subtask, Russian | 80 |
| Table 37: Intraclass Correlation Coefficients for Oral Reading Fluency | 81 |
| Table 38: Tajik Grade 2 National EGRA Mean Scores by Year | 91 |
| Table 39: Tajik Grade 4 National EGRA Mean Scores by Year | 91 |
| Table 40: Russian Grade 2 National EGRA Mean Scores by Year | 91 |
| Table 41: Russian Grade 4 National EGRA Mean Scores by Year | 92 |
| Table 42: Tajik Grade 2 National EGRA Zero Scores by Year | 92 |
| Table 43: Tajik Grade 4 National EGRA Zero Scores by Year | 92 |
| Table 44: Russian Grade 2 National EGRA Zero Scores by Year | 93 |
| Table 45: Russian Grade 4 National EGRA Zero Scores by Year | 93 |
| Table 46: Tajik Grade 2 National EGRA Grade-Level Fluency and Comprehension Proficiency by Year | 93 |
| Table 47: Tajik Grade 4 National EGRA Grade-Level Fluency and Comprehension Proficiency by Year | 93 |
| Table 48: Russian Grade 2 National EGRA Grade-Level Fluency and Comprehension Proficiency by Year | 94 |
| Table 49: Russian Grade 4 National EGRA Grade-Level Fluency and Comprehension Proficiency by Year | 94 |
| Table 50: Tajik Grade 2 2018 Mean Scores by School Group | 94 |
| Table 51: Tajik Grade 4 2018 Mean Scores by School Group | 94 |
| Table 52: Russian Grade 2 2018 Mean Scores by School Group | 95 |
| Table 53: Russian Grade 4 2018 Mean Scores by School Group | 95 |
| Table 54: Tajik Grade 2 2018 Zero Scores by School Group | 95 |
| Table 55: Tajik Grade 4 2018 Zero Scores by School Group | 96 |
| Table 56: Russian Grade 2 2018 Zero Scores by School Group | 96 |
| Table 57: Russian Grade 4 2018 Zero Scores by School Group | 96 |
| Table 58: Tajik Grade 2 2018 Grade-Level Fluency and Comprehension Proficiency by School Group | 97 |
| Table 59: Tajik Grade 4 2018 Grade-Level Fluency and Comprehension Proficiency by School Group | 97 |
| Table 60: Russian Grade 2 2018 Grade-Level Fluency and Comprehension Proficiency by School Group | 97 |
| Table 61: Russian Grade 4 2018 Grade-Level Fluency and Comprehension Proficiency by School Group | 97 |
| Table 62: Tajik Grade 2 2018 Mean Scores by Gender | 97 |
| Table 63: Tajik Grade 4 2018 Mean Scores by Gender | 98 |
| Table 64: Russian Grade 2 2018 Mean Scores by Gender | 98 |
| Table 65: Russian Grade 4 2018 Mean Scores by Gender | 98 |
| Table 66: Tajik Grade 2 2018 Zero Scores by Gender | 99 |
| Table 67: Tajik Grade 4 2018 Zero Scores by Gender | 99 |
| Table 68: Russian Grade 2 2018 Zero Scores by Gender | 99 |
| Table 69: Russian Grade 4 2018 Zero Scores by Gender | 100 |
| Table 70: Tajik Grade 2 2018 Grade-Level Fluency and Comprehension Proficiency by Gender | 100 |
| Table 71: Tajik Grade 4 2018 Grade-Level Fluency and Comprehension Proficiency by Gender | 100 |
| Table 72: Russian Grade 2 2018 Grade-Level Fluency and Comprehension Proficiency by Gender | 100 |
| Table 73: Russian Grade 4 2018 Grade-Level Fluency and Comprehension Proficiency by Gender | 101 |
| Table 74: Tajik Grade 2 2018 Mean Scores by Location | 101 |
| Table 75: Tajik Grade 4 2018 Mean Scores by Location | 101 |
| Table 76: Russian Grade 2 2018 Mean Scores by Location | 101 |
| Table 77: Russian Grade 4 2018 Mean Scores by Location | 102 |
| Table 78: Tajik Grade 2 Zero Scores by Location | 102 |
| Table 79: Tajik Grade 4 Zero Scores by Location | 103 |
| Table 80: Russian Grade 2 Zero Scores by Location | 103 |
| Table 81: Russian Grade 4 Zero Scores by Location | 103 |
| Table 82: Tajik Grade 2 2018 Grade-Level Fluency and Comprehension Proficiency by Location | 104 |
| Table 83: Tajik Grade 4 2018 Grade-Level Fluency and Comprehension Proficiency by Location | 104 |
| Table 84: Russian Grade 2 2018 Grade-Level Fluency and Comprehension Proficiency by Location | 104 |
| Table 85: Russian Grade 4 2018 Grade-Level Fluency and Comprehension Proficiency by Location | 104 |
| Table 86: Tajik Grade 2 2018 Mean Scores by Region | 104 |
| Table 87: Tajik Grade 4 2018 Mean Scores by Region | 105 |
| Table 88: Russian Grade 2 2018 Mean Scores by Region | 105 |
| Table 89: Russian Grade 4 2018 Mean Scores by Region | 105 |
| Table 90: Tajik Grade 2 2018 Mean Scores by Age | 106 |
| Table 91: Tajik Grade 4 2018 Mean Scores by Age | 106 |
| Table 92: Russian Grade 2 2018 Mean Scores by Age | 106 |
| Table 93: Russian Grade 4 2018 Mean Scores by Age | 107 |
| Table 94: Student Survey Item Frequencies by Language and Grade | 108 |
| Table 95: Tajik Student Survey Item Frequencies by Grade and School Group | 110 |
| Table 96: Russian Student Survey Item Frequencies by Grade and School Group | 112 |
| Table 97: Student Survey Mean and Composite Scores by Language and Grade | 114 |
| Table 98: Tajik Student Survey Mean and Composite Scores by School Group | 115 |
| Table 99: Russian Student Survey Mean and Composite Scores by School Group | 115 |
| Table 100: Student Background Factors Associated with Reading Outcomes by Grade and Language | 116 |
| Table 101: School Director Survey Item Frequencies | 117 |
| Table 102: School Director Interview Means | 118 |
| Table 103: Teacher Survey Item Frequencies | 119 |
| Table 104: Tajik Teacher Interview Means Overall | 124 |
| Table 105: Tajik Teacher Interview Means by Grade | 124 |
| Table 106: School Inventory Item Frequencies | 124 |
| Table 107: School Inventory Mean and Composite Scores | 125 |
| Table 108: Classroom Inventory Means and Composite Scores Overall | 125 |
| Table 109: Classroom Inventory Means and Composite Scores by Grade | 126 |
| Table 110: School Background Factors Associated with Associated with Reading Outcomes by Grade and Language | 127 |
List of figures
| Figure 1: National Oral Reading Fluency Results by Language, Grade, and Year | 15 |
| Figure 2: National Reading Comprehension Results by Language, Grade, and Year | 16 |
| Figure 3: 2018 Oral Reading Fluency Results by Language, Grade, and Gender | 16 |
| Figure 4: 2018 Oral Reading Fluency Results by Language, Grade, and Location | 17 |
| Figure 5: 2018 Oral Reading Fluency Results by Language, Grade, and Region | 17 |
| Figure 6: 2018 Oral Reading Fluency Results by Language, Grade, and School Group | 18 |
| Figure 7: 2018 Reading Comprehension Results by Language, Grade, and School Group | 18 |
| Figure 8: 2018 Zero Scores by Language, Grade, and Subtask | 19 |
| Figure 9: Tajik Grade 2 Oral Reading Fluency Mean Scores by Region, 2018 | 37 |
| Figure 10: Tajik Grade 4 Oral Reading Fluency Mean Scores by Region, 2018 | 38 |
| Figure 11: Russian Grade 2 Oral Reading Fluency Mean Scores by Region, 2018 | 39 |
| Figure 12: Russian Grade 4 Oral Reading Fluency Mean Scores by Region, 2018 | 40 |
| Figure 13: Tajik Grade 2 Mean Scores by School Group—Letter Name Identification, Familiar Word Reading, Nonword Reading, and Oral Reading Fluency, 2018 | 41 |
| Figure 14: Tajik Grade 2 Mean Scores by School Group—Initial Sound Identification, Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension, 2018 | 41 |
| Figure 15: Tajik Grade 4 Mean Scores by School Group—Familiar Word Reading, Nonword Reading, and Oral Reading Fluency, 2018 | 42 |
| Figure 16: Tajik Grade 4 Mean Scores by School Group—Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension, 2018 | 42 |
| Figure 17: Russian Grade 2 Mean Scores by School Group—Letter Name Identification, Familiar Word Reading, Nonword Reading, and Oral Reading Fluency, 2018 | 43 |
| Figure 18: Russian Grade 2 Mean Scores by School Group—Initial Sound Identification, Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension, 2018 | 43 |
| Figure 19: Russian Grade 4 Mean Scores by School Group—Familiar Word Reading, Nonword Reading, and Oral Reading Fluency, 2018 | 43 |
| Figure 20: Russian Grade 4 Mean Scores by School Group—Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension, 2018 | 44 |
| Figure 21: Percentage of RWM Intervention Students Reaching Reading Proficiency Benchmarks by Grade in 2018, Tajik | 45 |
| Figure 22: Percentage of RWM Intervention Students Reaching Reading Proficiency Benchmarks by Grade in 2018, Russian | 45 |
| Figure 23: Percentage of Students Speaking the Language of Instruction at Home by Grade, Language, and School Group | 47 |
| Figure 24: Percentage of Students Reporting that Their Mother Can Read the Language of Instruction by Grade, Language, and School Group | 47 |
| Figure 25: Percentage of Students Reporting that Their Father Can Read the Language of Instruction by Grade, Language, and School Group | 47 |
| Figure 26: Socioeconomic Status Index Score by Grade, Language, and School Group | 48 |
| Figure 27: Percentage of Students Reporting that They Attended Preschool by Language and Grade | 49 |
| Figure 28: Percentage of Students Reporting that They Had Books to Read at Their Home by Language and Grade | 49 |
| Figure 29: Percentage of Students Reporting That Their School Had Books to Take Home by Language and Grade | 50 |
| Figure 30: Teacher Expectations on the Grade Level for Students to Read Fluently and Write Simple Text | 51 |
| Figure 31: Distribution of Oral Reading Fluency Scores for All Students, Tajik Grade 2 | 82 |
| Figure 32: Distribution of Oral Reading Fluency Scores for All Students, Tajik Grade 4 | 82 |
| Figure 33: Distribution of Oral Reading Fluency Scores for All Students, Russian Grade 2 | 82 |
| Figure 34: Distribution of Oral Reading Fluency Scores for All Students, Russian Grade 4 | 83 |
| Figure 35: Tajik Grade 2 Mean Scores by Year—Letter Name Identification, Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 84 |
| Figure 36: Tajik Grade 2 Mean Scores by Year—Initial Sound Identification, Oral Vocabulary, Reading Comprehension, and Listening Comprehension | 84 |
| Figure 37: Tajik Grade 4 Mean Scores by Year—Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 84 |
| Figure 38: Tajik Grade 4 Mean Scores by Year—Oral Vocabulary, Reading Comprehension, and Listening Comprehension | 85 |
| Figure 39: Russian Grade 2 Mean Scores by Year—Letter Name Identification, Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 85 |
| Figure 40: Russian Grade 2 Mean Scores by Year—Initial Sound Identification, Oral Vocabulary, Reading Comprehension, and Listening Comprehension | 85 |
| Figure 41: Russian Grade 4 Mean Scores by Year—Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 86 |
| Figure 42: Russian Grade 4 Mean Scores by Year—Oral Vocabulary, Reading Comprehension, and Listening Comprehension | 86 |
| Figure 43: 2018 Tajik Grade 2 Mean Scores by Gender—Letter Name Identification, Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 86 |
| Figure 44: 2018 Tajik Grade 2 Mean Scores by Gender—Initial Sound Identification, Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension | 86 |
| Figure 45: 2018 Tajik Grade 4 Mean Scores by Gender—Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 87 |
| Figure 46: 2018 Tajik Grade 4 Mean Scores by Gender—Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension | 87 |
| Figure 47: 2018 Russian Grade 2 Mean Scores by Gender—Letter Name Identification, Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 87 |
| Figure 48: 2018 Russian Grade 2 Mean Scores by Gender—Initial Sound Identification, Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension | 87 |
| Figure 49: 2018 Russian Grade 4 Mean Scores by Gender—Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 88 |
| Figure 50: 2018 Russian Grade 4 Mean Scores by Gender—Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension | 88 |
| Figure 51: 2018 Tajik Grade 2 Mean Scores by Location—Letter Name Identification, Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 88 |
| Figure 52: 2018 Tajik Grade 2 Mean Scores by Location—Initial Sound Identification, Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension | 88 |
| Figure 53: 2018 Tajik Grade 2 Mean Scores by School Group and Location—Oral Reading Fluency | 89 |
| Figure 54: 2018 Tajik Grade 4 Mean Scores by Location—Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 89 |
| Figure 55: 2018 Tajik Grade 4 Mean Scores by Location—Oral Vocabulary, Reading Comprehension, Silent Reading Comprehension, and Listening Comprehension | 89 |
| Figure 56: 2018 Tajik Grade 4 Mean Scores by School Group and Location—Oral Reading Fluency | 89 |
| Figure 57: 2018 Russian Grade 2 Mean Scores by Location—Letter Name Identification, Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 90 |
| Figure 58: 2018 Russian Grade 2 Mean Scores by Location—Initial Sound Identification, Oral Vocabulary, Reading Comprehension, and Listening Comprehension | 90 |
| Figure 59: 2018 Russian Grade 4 Mean Scores by Location—Familiar Word Reading, Nonword Reading, and Oral Reading Fluency | 90 |
| Figure 60: 2018 Russian Grade 4 Mean Scores by Location—Oral Vocabulary, Reading Comprehension, and Listening Comprehension | 90 |
Executive Summary
Study Purpose This is the baseline report of a five-year study that seeks to understand trends in early grade reading performance in Tajikistan at the national level and among schools benefiting from the United States Agency for International Development (USAID) Read with Me (RWM) project. Findings from this study aim to inform implementation of RWM and support early grade reading in Tajikistan.
Project Background RWM is a five-year project implemented by Chemonics International, Inc. that aims to improve reading outcomes for 75 percent of all primary grade students nationwide over the course of implementation. It is designed to achieve this goal through improved reading instruction in grades 1 through 4, improved access to and availability of quality materials that support the development of reading in grades 1 through 4, increased innovations and partnerships supporting literacy outcomes, and increased government support to improve reading.
RWM is the latest USAID-funded project intervening nationwide in primary education in Tajikistan. It builds on the gains made most recently by two previous education projects: the 2007–12 Quality Learning Project (QLP) and the 2013–17 Quality Reading Project (QRP).
design, research questions, methods and limitations study design and research questions This is a five-year study with data collection taking place in spring 2018 as a baseline, spring 2019 as a midline, and spring 2021 as an endline.
The first goal of this study is to understand national-level trends in early grade reading performance. To this end, the study compares data collected under the predecessor project—QRP—with data collected by RWM.
The second goal of this study is to estimate RWM’s intervention effects on reading progress at the primary level by comparing performances between three distinct populations of schools in Tajikistan:
1. QRP-only: public schools with primary grades that received QRP interventions and will not receive RWM interventions
2. QRP+RWM: public schools with primary grades that received QRP interventions and will receive RWM interventions
3. RWM-only: public schools with primary grades that did not receive QRP interventions and will receive RWM interventions
Based on these two goals, this study seeks to address five research questions:
1. Nationally, how do Tajik and Russian reading outcomes of students in grade 2 and grade 4 vary by subgroups and across time points?[footnoteRef:2] [2: For this study, subgroups of interest include gender, location (urban or rural), and region.]
2. Which RWM geographic subgroups require supplementary attention? And, what kind of supplementary attention is required?
3. How do Tajik and Russian reading outcomes of students in grade 2 and grade 4 differ between RWM-only and QRP-only schools across time points?
4. How do Tajik and Russian reading outcomes of students in grade 2 and grade 4 differ between RWM-only and QRP+RWM schools across time points?
5. At each time point, what proportion of students in grade 2 and grade 4 can read and understand the meaning of grade-level text in Tajik and in Russian in schools served by RWM?
Methods To answer the research questions, RWM drew a representative sample of each of the three school groups for data collection at baseline in 2018. The sample represents the full population of schools in Tajikistan based on region, language of instruction (Russian and Tajik), and location (urban and rural). Students were sampled at the class level by gender.
Students were assessed using Tajik and Russian adaptations of the Early Grade Reading Assessment (EGRA), which measures student performance on the basic foundational skills required for fluency in reading. Alongside the EGRA, students took brief survey that collected information about their background and home environment. Additionally, school-level data were collected using Snapshot of School Management Effectiveness (SSME) surveys, including a school director questionnaire, a teacher questionnaire, a school inventory checklist, and a classroom inventory checklist. The purpose of the student survey and SSME data was to understand student and school background factors that may influence students’ reading performance.
Baseline data collection took place in April and May 2018. Data were collected by enumerators, many of whom were suggested by the MoES and hired by M-Vector, a local data collection firm. In total, enumerators visited 287 schools—217 in the Tajik sample and 70 in the Russian sample (given the proportionate population sizes); they collected data from 2,811 grade 2 students—2,123 in the Tajik sample and 688 in the Russian sample—and 2,665 grade 4 students—2,093 in the Tajik sample and 572 in the Russian sample. Details of sampling and the census approach are found in Methods and Limitations.
Prior to analysis, the EGRA tools were equated with—or brought to a common scale as—the EGRA tools used by QRP. This equating was done to allow for appropriate comparisons with QRP data from previous years. Sampling weights were also calculated and applied to the assessment data to produce more representative estimates in the sample of students. After applying the equating and weighting functions to the 2018 data, mean scores, zero scores, and reading benchmarks, results across subtasks were analyzed for each of the disaggregated groups of interest. Differences between within-group categories were analyzed using t-tests for mean score comparisons and chi-square tests for zero score and reading benchmark comparisons.
Data for the SSME questionnaires and inventories were correlated with the oral reading fluency (ORF) subtask scores to identify student and school characteristics that are linked to the subtask scores. Those items with statistically significant correlations with the ORF subtask scores were included in a linear regression model to understand the extent to which they predicted the ORF subtask scores.
limitations The following are key limitations that should be kept in mind while reviewing EGRA results:
1. Most schools in Tajikistan have participated in a previous USAID-funded reading intervention (QRP), and all eligible schools that did not participate in QRP are receiving RWM interventions. Therefore, an appropriate comparison population—or control group—of schools does not exist in the country. As this baseline report finds that differences exist between schools that participated in QRP and schools that will participate in RWM even before the intervention reaches Cohort 2 schools, future differences in performance at midline and endline may not necessarily be attributed to RWM.
2. As with most EGRA studies, the results are not appropriate data for cross-linguistic comparisons. That is, the report does not statistically compare students’ reading skills in Tajik with students’ reading skills in Russian. Acquisition of language and reading development depends on several factors including the different levels of orthographic transparency, visual complexity, and phonology.
Findings Key findings from the report are presented below. More detailed results are presented in the report’s EGRA Results and Contextualizing EGRA Results sections. See Annexes H-K for comprehensive results from the EGRA and SSME.
National-Level Results Snapshot Overall, 2018 EGRA results for students studying in Tajik and Russian show strong reading outcomes (see Figure 1 and Figure 2). The ORF subtask mean scores for grade 2 students in Tajik-language schools and grade 4 students in Russian-language schools approached the established benchmarks for their respective grades. Regardless of language or grade, the percentage of students who were nonreaders across most EGRA subtasks was extremely low—less than five percent—indicating that nearly all students have some ability to read.
For Tajik-language students, their relative performance on the 2018 EGRA versus previous years’ EGRAs varied depending on the EGRA subtask and the year being compared. For grade 2 students, 2018 results were comparable to or higher than 2014 and 2017 results but lower than 2016 results across most subtasks. For grade 4 students, 2018 results were generally comparable to 2014 results but lower than 2016 or 2017 results.
For Russian-language students, their relative performance on the 2018 EGRA versus previous years’ EGRAs varied by grade only. For grade 2 students, the 2018 results were lower than those from 2014, 2016, or 2017 across all subtasks. For grade 4 students, however, 2018 results were comparable or higher than 2014, 2016, or 2017 results across most subtasks.
Figure 1: National Oral Reading Fluency Results by Language, Grade, and Year
Note: Languages and grades marked with the letter “a” denotes differences between 2018 and 2014 are statistically significant at p < 0.05. Languages and grades marked with the letter “b” denotes differences between 2018 and 2016 are statistically significant at p < 0.05. Languages and grades marked with the letter “c” denotes differences between 2018 and 2017 are statistically significant at p < 0.05.
Figure 2: National Reading Comprehension Results by Language, Grade, and Year
Note: Languages and grades marked with the letter “a” denotes differences between 2018 and 2014 are statistically significant at p < 0.05. Languages and grades marked with the letter “b” denotes differences between 2018 and 2016 are statistically significant at p < 0.05. Languages and grades marked with the letter “c” denotes differences between 2018 and 2017 are statistically significant at p < 0.05.
2018 subgroup results by gender, location, and region Girls generally outperformed boys. Among Tajik-language grade 2 and grade 4 students and Russian-language grade 2 students, girls typically scored higher than boys on timed subtasks, such as letter name identification, familiar word reading, nonword reading, and ORF (see Figure 3 for ORF results).
Figure 3: 2018 Oral Reading Fluency Results by Language, Grade, and Gender
Note: An asterisk (*) denotes differences between girls and boys are statistically significant at the p < 0.05 level.
Students from urban areas generally outperformed students from rural areas. This was particularly true for Tajik-language schools[footnoteRef:3], where students from urban areas outperformed students from rural areas on nearly every subtask (see Figure 4 for ORF results). [3: The population of Russian schools was imbalanced in favor of urban schools/students. For Grade 2 Russian, there were enough rural students and the differences were large enough to detect statistically significant difference on most tasks. For Russian Grade 4, the sample was too small to detect statistically significant differences across all tasks except oral vocabulary. See Table 11.
Figure 4: 2018 Oral Reading Fluency Results by Language, Grade, and Location
Note: An asterisks (*) denotes differences between students from rural and urban areas are statistically significant at the p < 0.05 level.
Students from schools located in Dushanbe and the Gorno-Badakhshan Autonomous Oblast (GBAO) generally outperformed students from the Districts of Republican Subordination (DRS), Khatlon-Bokhtar, Khatlon-Kulob, and Sughd. In Tajik-language schools, students from Dushanbe and GBAO typically had the highest mean scores relative to other regions. In Russian-language schools, Dushanbe typically had the highest mean scores relative to other regions. These results were consistent across grades.
Figure 5: 2018 Oral Reading Fluency Results by Language, Grade, and Region
Tajik Grade 2
Tajik Grade 4
Russian Grade 2
Russian Grade 4
Note: The Russian sample did not include schools from GBAO because Russian-language schools were not present in this region at the time of sampling or data collection. Additionally, results from Khatlon-Kulob are not reported due to insufficient sample size.
2018 school group results For Tajik-language students, 2018 EGRA results for the RWM-only group were generally comparable or higher than those for the other two groups (see Figure 6 and Figure 7). Grade 2 students from the RWM-only group had similar or higher mean scores than students from the other two groups across most subtasks. Among grade 4 students, the RWM-only group had similar mean scores as students from the other two groups across most subtasks. At the time of data collection, RWM had not begun implementation in these schools. Differences in performance are due to variations in the school populations and are not attributable to RWM.
For Russian-language students, 2018 EGRA results varied by grade and school group (see Figure 6 and Figure 7). For grade 2 students, the RWM-only had lower mean scores than the QRP+RWM group across all subtasks.[footnoteRef:4] For grade 4 students, mean scores were comparable across the RWM-only and QRP+RWM groups on nearly all subtasks. As with Tajik-language results, these results reflect differences in the school populations served by RWM and QRP rather than RWM interventions, which had not yet begun at the time of data collection. [4: Due to the small number of QRP-only students in the Russian sample, Russian school group findings only compare the performance of the students in the RWM-only group with the RWM+QRP group.]
Figure 6: 2018 Oral Reading Fluency Results by Language, Grade, and School Group
Note: The QRP-only group was excluded from comparisons between groups due to an insufficient sample size—30 students for grade 2 and 10 students for grade 4. Subtasks marked with the letter “a” denotes differences between the RWM-only group and the QRP-only group are statistically significant at p < 0.05. Subtasks marked with the letter “b” denotes differences between the RWM-only group and the QRP+RWM group are statistically significant at p < 0.05.
Figure 7: 2018 Reading Comprehension Results by Language, Grade, and School Group
Note: The QRP-only group was excluded from comparisons between groups due to an insufficient sample size—30 students for grade 2 and 10 students for grade 4. Subtasks marked with the letter “a” denotes differences between the RWM-only group and the QRP-only group are statistically significant at p < 0.05. Subtasks marked with the letter “b” denotes differences between the RWM-only group and the QRP+RWM group are statistically significant at p < 0.05.
Student and School Background Factors At the student level, several factors had positive associations with reading outcomes, including parents’ ability to read in the language of instruction (LOI), the availability of books to read at home, the ability to take books home from school for reading for pleasure (i.e not text books), and preschool attendance.
At the school level, several factors had positive associations with reading outcomes, including the availability of student resources, class size, and teacher professional development on reading instruction.
Conclusions and recommendations Among the study findings, several conclusions stand out as having implications for RWM implementation.
Students’ comprehension skills generally are not at the same level as their fluency skills. The national mean score for grade 2 Tajik-language students on the ORF subtask was 39.43 correct words per minute (CWPM), a mark just below the national benchmark; yet students answered less than 40.0 percent of the five reading comprehension questions correctly. Moreover, the percentage of students who did not answer a single question correctly—known as zero scores—on the reading comprehension subtasks were among the highest of all subtasks ( Figure 8). To address this skill gap, targeted reading interventions are needed to help improve the reading comprehension for students across both languages and grades. Students need greater support to learn how to comprehend text. The RWM intervention should be cognizant of the need to develop teachers’ understanding of how to build students’ explicit reading comprehension skills through strategies that focus on vocabulary, listening comprehension, and overall exposure to a variety of types of narrative.
Figure 8: 2018 Zero Scores by Language, Grade, and Subtask
Tajik Grade 2
Russian Grade 2
Tajik Grade 4
Russian Grade 4
A second key finding with implications for RWM implementation is that there is a performance gap between girls and boys. EGRA results from 2018 showed that girls generally scored higher than boys did on most subtasks; these results are consistent with previous EGRAs conducted in Tajikistan. To the extent possible, future teacher training should be mindful of the difference in reading outcomes between boys and girls and provide strategies to narrow this gap.
A third key finding with implications for RWM implementation is that there is a relationship between student performance and where a student lives. Students from urban areas typically outperformed their rural peers. Differences by region also emerged, as students from Dushanbe and GBAO generally outperformed students from other regions. This trend by region suggests that there may be regional differences that are contributing to differences in EGRA performance. Extra support to teachers and schools in rural areas—especially in the DRS, Khatlon-Bokhtar, Khatlon-Kulob, and Sughd—would be helpful to ensure that these students do not continue to underperform when compared with their peers in urban areas and in the GBAO.
A fourth key finding with implications for RWM implementation is the importance of fostering a culture of reading among students. At the student level, the presence of books at home and the ability to bring books home all had positive associations with reading performance. To the extent possible, work should be done to promote and provide opportunities for students to read outside of school.
Introduction
RWM project background The Government of the Republic of Tajikistan has implemented reforms to improve the quality of its education system since independence and country’s five-year civil war in the 1990s.[footnoteRef:5] The Government of Tajikistan and the Ministry of Education and Science (MoES) view inclusive access to quality education as a requirement of the country’s transition to a market economy and is committed to reforming the education sector as outlined in its national strategy for education development (2012–20) and midterm development strategy (2016–20). Investment in education—particularly early childhood education and the inclusion of girls, rural students, and students with disabilities—supports building the capacity of Tajikistan’s workforce and contributing to the economic development of the country.[footnoteRef:6] Although Tajikistan’s education budget has steadily increased as a percentage of the gross domestic product, the level of financing is inadequate to meet system needs, according to a funding gap analysis conducted by the MoES.[footnoteRef:7] [5: Government of the Republic of Tajikistan, National Strategy for Education Development for 2012-2020. (2012), 3-4. http://planipolis.iiep.unesco.org/sites/planipolis/files/ressources/tajikistan_ed_sector_plan_2012-2020.pdf and United Nations in Tajikistan. Annual UNDAF Results Report. (2016). https://untj.org/files/Publications/UNDAF/UNDAF_Annual_Report_English_Version_2016.pdf] [6: Government of the Republic of Tajikistan, National Strategy for Education Development for 2012-2020. (2012), 3, 9-12.] [7: The World Bank Group, Tajikistan Partnership Program Snapshot (October 2015), 5. http://pubdocs.worldbank.org/en/645741444794465533/Tajikistan-Snapshot.pdf ]
Aid agencies and international organizations in Tajikistan help to fill this gap through initiatives like early childhood education, inclusive education, quality improvements in education, and child-friendly classrooms, as well as other educational programming. The United States Agency for International Development (USAID) has provided support to the government’s reforms to primary education for years through efforts such as the 2007–12 Quality Learning Project (QLP) and the 2013–17 Quality Reading Project (QRP).[footnoteRef:8] USAID is now supporting the third initiative—the Read With Me project (RWM)—to build on the gains made by these two projects in reading skills, including comprehension. [8: RWM expands on QRP’s work, which reached over 60 percent of schools in Tajikistan. QPR supported reading instruction in grade 1 through 4 through in-service training and classroom-based mentoring for educators, as well as Tajik and Russian-language reading materials, and other school-based reading activities and governmental policy support to reading. See American Institutes for Research, USAID Quality Reading Project Republic of Tajikistan: Final EGRA and Impact Report, 2013–2017, (2017).
RWM is a five-year (2016–21) project that builds on both QLP and QRP, as well as other education programs in Tajikistan; it supports the National Strategy for Education Development 2012–20 and the Midterm Development Strategy for 2016–20. USAID’s RWM project will not only involve the MoES and schools to support the improvement of reading outcomes, but it will also reach out to communities as well as the private sector. It looks to introduce the role of science, technology, innovations, and partnership in supporting reading outcomes. The RWM project will provide supports directly to schools in a series of five cohorts from 2018 through 2021.
RWM aims to improve reading outcomes for students in grades 1 through 4 in targeted schools in Tajikistan by
a. Increasing the availability of age-appropriate reading materials in Tajik and Russian languages, which will develop students’ core reading skills;
b. Providing educators with reading-specific in-service training, supplementary materials, and the integration of assessments to monitor reading progress;
c. Increasing innovations and partnerships supporting literacy outcomes; and
d. Increasing government support for reading through capacity building in reading instruction and mentoring as well as increasing dialogue on the reading environment.[footnoteRef:9] [9: Chemonics International, Inc, USAID Read With Me Project Annual Report Year 1. (2017). https://pdf.usaid.gov/pdf_docs/PA00N7RF.pdf ]
RWM aims to support 75 percent of all primary school teachers nationwide (approximately 24,000 teachers) and 75 percent of all primary grade students nationwide (approximately 524,000 students) at approximately 2,800 schools over the course of implementation. RWM is active in every region of Tajikistan and expands USAID’s support of early grade reading education to the Gorno-Badakhshan Autonomous Oblast (GBAO).
Early grade reading outcomes in Tajikistan The RWM Early Grade Reading Assessment (EGRA) builds on a foundation of research in early grade reading outcomes in Tajikistan. Previous USAID-funded projects have collected a wealth of data through EGRA and provide the most extensive research into early grade reading outcomes in Tajikistan. In 2011, the first USAID-funded EGRA in Tajikistan analyzed results from students in Tajik and Russian in grades 2 to 4 and included a qualitative study of early grade reading pedagogy in Tajikistan.[footnoteRef:10] This study, implemented during QLP, found that students generally had strong foundations in early grade reading skills, including the alphabetic principle and recognition of sight words; however, they had a weakness in decoding unfamiliar words and comprehending texts. The report suggested links between classroom practices and these outcomes, as pedagogy at that time focused more on rote memorization of words and speed reading rather than on phoneme segmentation, decoding skills, and comprehension strategies.[footnoteRef:11] [10: Mirka Tvaruzkova and Duishon Shamatov, Review of Early Grade Reading Teaching and Skills: The Kyrgyz Republic and Tajikistan Final Report. (Washington, DC: United States Agency for International Development, 2012).] [11: The focus of the interviews was on current pedagogical practices and materials used in early grade reading instruction, as well as factors that affect early grade reading acquisition. Interview guides were prepared and used in a semi-structured format with stakeholder groups. A total of 25 interviews were conducted in Tajikistan. The sample included education officials from the MoES, preservice teacher training institution staff, Academy of Education members, authors of children’s literature, representatives of international organizations, primary school teachers, and parents.]
QRP’s interventions were designed to address these areas of improvement by drawing on predictors of reading success identified by the EGRA, such as the promotion of a “culture of literacy” with parents and the publication of grade-level texts. In addition to program interventions, QRP collected EGRA data for baseline assessments in 2014, midline assessments in 2016, and endline assessments in 2017 as part of a randomized control trial in primary grades in almost all regions of the country, excluding GBAO. Early QRP results were consistent with the 2011 EGRA report.[footnoteRef:12] [12: Mirka Tvaruzkova and Duishon Shamatov, Review of Early Grade Reading Teaching and Skills: The Kyrgyz Republic and Tajikistan Final Report. (Washington, DC: United States Agency for International Development, 2012).]
QRP reported overall trends of growth in Tajik- and Russian-reading outcomes from 2014 to 2016, with declines in 2017 that reverted to 2014 levels. Overall, these results suggest that by 2017, students in Tajikistan generally demonstrated foundational reading skills. Phoneme segmentation and dictation appear to have improved from 2011. The rate of zero scores, or students who were unable to complete a subtask, was low across all grades, languages, and subtasks. Overall, the oral reading fluency (ORF) subtask scores fluctuated over the course of QRP, with less than half of students in Tajik- or Russian-language schools attaining proficiency scores by 2017. The reading comprehension subtask scores, while showing gains among students in grade 4 who benefitted from QRP interventions, remained fairly low; a large number of students performing poorly.[footnoteRef:13] [13: American Institutes for Research, USAID Quality Reading Project Republic of Tajikistan: Final EGRA and Impact Report, 2013–2017. (2017). ]
Purpose of the study This report provides baseline data to examine reading trends at the national level and among RWM schools. The report uses data collected by QRP to provide nationally representative results over time. The report also describes data collected by RWM in April 2018 from three distinct populations of schools in Tajikistan to provide baseline values for student reading outcomes in schools that have yet to participate in RWM. Comparisons with baseline at midline and endline will provide estimates of RWM intervention effects on reading progress at the primary level by comparing performance between populations receiving and not receiving RWM. To this end, the RWM EGRA gathers information about three populations of schools:
1. QRP-only: public schools with primary grades that received QRP interventions and will not receive RWM interventions
2. QRP+RWM: public schools with primary grades that received QRP and will receive RWM interventions
3. RWM-only: public schools with primary grades that did not receive QRP interventions and will receive RWM interventions
In short, this baseline report serves two goals:
1. To provide a nationally representative “snapshot” of reading performance in Tajikistan from April 2014 to April 2018; and
1. To estimate differences in reading performance between students in the three groups of schools that received QRP interventions or are receiving RWM interventions.
The RWM project collected baseline data in April through May 2018 (while the academic school year runs from September to June[footnoteRef:14]). The midline data collection will take place in spring 2019 and the endline data collection in spring 2021. Data from these collection periods will answer specific research questions about reading performance in Tajikistan over time and about RWM itself. [14: The scale of the RWM implementation (training more than 800 schools at a time) demands a four level cascade (national trainers train regional trainers, who train three school-level trainers who then train the rest of the teachers in their school over a period of up to 16 weeks (one day per week) hence some teachers may not receive the full course until the end of the first term (September to December).]
To meet these goals, student, teacher, and school data are reported using the following tools: the EGRA, which measures student performance on the basic foundational skills required for fluency in reading; a student survey, which captures background information about the student being assessed; and Snapshot of School Management Effectiveness (SSME) surveys, which capture teachers’ and administrators’ perceptions of school management and pedagogic practices in the school.[footnoteRef:15] [15: The SSME was designed to capture indicators that are believed to affect student learning, and the results of these tools can inform education stakeholders about current practices in schools and classrooms in Tajikistan. ]
Research questions This baseline EGRA report seeks to answer five research questions:
1. Nationally, how do Tajik and Russian reading outcomes of students in grade 2 and…
This is the start of the file's text. The full file is on GovTribe.
File details come from the government source that posted it. Updated .