USAID Girls Leadership and Empowerment through Education Final Performance Evaluation Report.pdf

GIRLS LEADERSHIP AND

EMPOWERMENT THROUGH

EDUCATION

FINAL PERFORMANCE

EVALUATION REPORT

August 2023

This publication was produced at the request of the United States Agency for International Development. It was prepared independently by EdIntersect and School-to-School International for Winrock International.

1 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

GIRLS LEADERSHIP AND EMPOWERMENT

THROUGH EDUCATION

FINAL PERFORMANCE EVALUATION

August 28, 2023

PROJECT NUMBER 72068818C00001; SUBCONTRACT NUMBER 6857-23-C-01

DISCLAIMER

This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of Winrock International and do not necessarily reflect the views of USAID or the United States Government.

USAID.GOV USAID GLEE FINAL PERFORMANCE EVALUATION REPORT | 2

CONTENTS

Table of Figures 3

Table of Tables 5

Acronyms 6

Executive Summary 7 Evaluation Purpose 7 Project Background 7 Evaluation Methodology Overview 8 Answering The Evaluation Questions 8 Conclusions and Recommendations 13

Introduction 16 Context and Project Background 16 Evaluation Purpose and Audience 17

Evaluation Questions and Methodology Overview 19

Evaluation Questions 19 Methodology Overview 20

Sample Description 25

Quantitative Sample 25 Qualitative Sample 26

Findings 27

Relevance 27 Effectiveness 30 Impact 57 Efficiency 61 Sustainability 64

Limitations 66

Conclusions and Recommendations 67

Annexes 71

Annex I: Evaluation Statement of Work 71 Annex II: Evaluation Methods and Limitations 72 Annex III: Final Evaluation Tools 76

3 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

Annex IV: Additional Analyses 144

TABLE OF FIGURES

Figure 1. Proportion of Girls Reporting Having the Opportunity to Participate in Awareness-raising Sessions on Girls' Education

Figure 2. Proportion of Girls Reporting Having Responsibilities Outside of School

Figure 3. Proportion of Girls Reporting that Responsibilities Outside of School Sometimes Prevent Them from Studying at Home or Going to School

Figure 4. Proportion of Girls Who Agree Girls and Boys Have an Equal Right to Attend School

Figure 5. Proportion of Girls Who Agree Girls Should Stay in School When They Get Married, Compared with Boys

Figure 6. Proportion of Girls Who Agree Girls Should Stay in School When They Have a Child, Compared with Boys

Figure 7. Proportion of Girls Reporting Having the Opportunity to Take Part in Awareness-raising Sessions on Gender-based Violence and School Security

Figure 8. Proportion of Girls’ Responses to the Statement, “Girls are safe at school.”

Figure 9. Proportion of Girls’ Responses to the Statement, “Boys are safe at school.”

Figure 10. Proportion of Girls’ Responses to the Statement, “Teachers do not have the right to touch students’ thighs, behinds, or private parts.”

Figure 11. Proportion of Girls’ Responses to the Statement, “Teachers at my school touch students’ thighs, behinds, or private parts.”

Figure 12. Proportion of Girls’ Responses to the Statement, “Teachers at my school demand to have sexual relations with certain girls.”

Figure 13. Proportion of Girls’ Responses to the Statement, “It is sometimes a girl’s fault if a teacher sexually harasses her.”

Figure 14. Proportion of Girls’ Responses to the Statement, “It is sometimes a girl’s fault if a student sexually harasses her.”

Figure 15. Proportion of Girls’ Responses to the Statement, “It is sometimes a girl’s fault if a teacher touches her thighs, behind, or private parts.”

Figure 16. Proportion of Girls’ Responses to the Statement, “It is sometimes a girl’s fault if a student touches her thighs, behind, or private parts.”

Figure 17. Proportion of Girls Reporting Knowing Someone in Their Class Who Has Used the Incident Box This Year

Figure 18. Reasons Reported by Girls for Why More Students Do Not Use the Incident Box

Figure 19. Proportion of Girls Reporting They Have Started Their Periods

Figure 20. Proportion of Girls Reporting Knowing Where to Get Information on Menstrual Cycles

USAID.GOV USAID GLEE FINAL PERFORMANCE EVALUATION REPORT | 4

Figure 21. Proportion of Girls Reporting Being Comfortable Going to School During Their Periods

Figure 22. Proportion of Girls Reporting Having Stayed at Home Instead of Going to School When on Period

Figure 23. Proportion of Girls’ Responses to the Statement, “I’m ashamed of my body when I have my period.”

Figure 24. Proportion of Girls’ Responses to the Statement, “It is important that I keep my period a secret.”

Figure 25. Proportion of Girls’ Responses to the Statement, “I’m proud of having my period.”

Figure 26. Proportion of Girls’ Responses to the Statement, “Getting my period is no big deal for me.”

Figure 27. Material Reported by Girls Used to Manage Period

Figure 28. Proportion of Girls Reporting They Have Not Had Access to Material in the Past Three Months

Figure 29. Proportion of Girls Reporting Having Seen or Heard Family Planning and Reproductive Health Messaging in the Current School Year

Figure 30. Proportion of School Directors Reporting that Their Schools Held Sessions on Reproductive Health Topics

Figure 31. Source of Family Planning and Reproductive Health Messaging

Figure 32. Family Planning Methods Girls Reported Having Heard of During the 2022–23 School Year . 47

Figure 33. Reproductive Health Messages Girls Reported Having Heard During the 2022–23 School Year

Figure 34. Proportion of Girls’ Responses to the Statement, “I know where to go if I need contraception.”

Figure 35. Proportion of Girls’ Responses to the Statement, “I would be too shy or uncomfortable to go to a clinic or center to get contraception.”

Figure 36. Proportion of Girls Reporting Having the Opportunity to Participate in Awareness-raising Sessions on Reproductive Health, COVID-19, Family Planning, Menstrual Hygiene, and Forced or Early Marriage

Figure 37. Proportion of School Directors Reporting Forced or Early Marriage, COVID-19, and Menstrual Hygiene Awareness-raising Sessions Being Held at Their School

Figure 38. Proportion of Timing of Most Recent Awareness-building Sessions

Figure 39. Facilitators of Menstrual Hygiene Sessions, As Reported by School Directors

Figure 40. Proportion of Girls’ Correct Responses to Menstrual Hygiene Questions on the BBS

Figure 41. Proportion of Girls Reporting Knowing What Periods or Menses Are

Figure 42. Proportion of Girls’ Responses on the Average Length of the Menstrual Cycle, From Those Who Reported Starting Their Period and Reported Knowing the Average Length

Figure 43. Accessibility and Use of Handwashing Facilities

5 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

Figure 44. Proportion of Enumerators Reporting Water Available for Washing Hands at the Time of the School Visit

Figure 45. Proportion of Enumerators Reporting Soap or Ashes Available for Washing Hands at the Time of the School Visit

Figure 46. Proportion of Girls’ Reported Benefits of the USAID GLEE Project

Figure 47. Proportion of Girls’ Reported USAID GLEE Activities Enabled Them to Feel Safer at School or on the Way to School

Figure 48. Proportion of Girls’ Reported USAID GLEE Activities Enabled Them to Attend School More Regularly

TABLE OF TABLES

Table 1. Life-of-Project Data Collection Timeline, Sampling Approach, and Instruments Used

Table 2. Evaluation Questions

Table 3. Mapping Mixed-Methods Data Sources to Indicators and Evaluation Domains

Table 4. Beneficiary-based Survey Sample Design

Table 5. Target Qualitative Sample

Table 6. Actual Quantitative Sample

Table 7. Girls’ Quantitative Sample, by School Type and Age

Table 8. Actual Qualitative Sample

Table 9. Girls Reporting Missing School Due to Safety Issues

Table 10. Girls Reporting Knowing What Menses Is

Table 11. Girls Reporting Knowing the Average Length of Menstrual Cycle

Table 12. Schools that Provide Hygiene Lessons for All Students

Table 13. Presence of Handwashing Facilities at Schools

Table 14. Summary of USAID GLEE Project Indicators Calculated as Part of the Final Performance Evaluation

Table 15. Topics of Stories of Most Significant Change Shared by Girls in FGDs

USAID.GOV USAID GLEE FINAL PERFORMANCE EVALUATION REPORT | 6

ACRONYMS

AME Association des mères d’élèves (Mothers’ group)

AMELP Activity monitoring, evaluation, and learning plan

APE Association des parents d’élèves (Parents’ group)

ASC Accelerated schooling centers (Centre de scolarisation accéléré)

BBS Beneficiary-based survey

CERIPS Centre d’Etude et de Recherche sur l’Information en Population et Santé

CGS School management committee (Comités de gestion scolaire)

COVID-19 Coronavirus Disease 2019

CPHDA Centre d’assistance et de promotion des droits humains en Afrique

EQ Evaluation question

FGD Focus group discussion

FP Family planning

GAAS Groupe d’Animation, Action, au Sahel Mali

GBV

GLEE

Gender-based violence

Girls Leadership and Empowerment through Education

GoM Government of Mali

KII Key informant interview

MSC Most Significant Change

NGO Non-governmental organization

OMAES Œuvre Malienne d'Aide à l'Enfance du Sahel

RH Reproductive health

SR Sub-result

SRGBV School-related gender-based violence

STS School-to-School International

U.S. United States of America

USAID United States Agency for International Development

USAID GLEE

USG

USAID Girls Leadership and Empowerment through Education

United States government

WASH Water, sanitation, and hygiene

WI Winrock International

7 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

EXECUTIVE SUMMARY

EVALUATION PURPOSE

In May 2023, EdIntersect, with its partners School-to-School International (STS) and Centre d’Etude et de Recherche sur l’Information en Population et Santé (CERIPS), conducted data collection for a mixed-methods final performance evaluation of the USAID Girls Leadership and Empowerment through Education (USAID GLEE) project in Mali. The final performance evaluation sought to assess the project’s achievements as outlined in the results framework; assess its relevance, effectiveness, efficiency, impact, and sustainability; and listen to and engage with girls as key informants on USAID GLEE’s outcomes.

The evaluation engaged both current and previous project stakeholders and beneficiaries. Data collection was conducted in the Bandiagara and Douentza regions, where USAID GLEE is currently implemented, and in the Kayes region, where USAID GLEE concluded activities in 2021. Results from the final performance evaluation will be shared with USAID, project staff, partner organizations, and other key stakeholders in Mali to use when considering future activities.

PROJECT BACKGROUND

USAID GLEE project is a five-year, USAID-funded project to increase girls’ educational opportunities in Mali by supporting and leveraging existing efforts by the Government of Mali (GoM) and civil society.

Launched in 2018 and concluding in 2023, the project works to decrease barriers to accessing quality education, improve girls’ safety, and increase girls’ knowledge and adoption of positive health behaviors to increase access to education for adolescent girls and enable them to obtain greater educational attainment.

The USAID GLEE project employed seven key strategies to meet these objectives:

1. Community mobilization through school management committees (comités de gestion scolaires, CGSs), parents’ groups (association des parents d’élèves, APEs), and other key community stakeholders

2. Participatory capacity-building of accelerated schooling centers (centres de scolarisation accéléré, ASC) facilitators and teachers at partner schools

3. Mentorship and peer learning through USAID GLEE mentors, youth ambassadors, and grandmothers

4. Targeted social behavior change communications around girls’ education, safety, and health

5. Payment of girls’ scholarships

6. Provision of teaching and learning materials

7. Improving water, sanitation, and hygiene (WASH) facilities at schools

Winrock International (WI) leads the USAID GLEE consortium of four partners—Groupe d’Animation, Action au Sahel Mali (GAAS), Centre d'assistance et de promotion des Droits Humains en Afrique (CPHDA), Œuvre Malienne d'Aide à l'Enfance du Sahel (OMAES), and IntraHealth International. GAAS and CPHDA implement the community engagement and school support activities, OMAES is responsible for the ASCs and teacher professional development activities, and IntraHealth International implements the reproductive health (RH) and WASH activities.

USAID.GOV USAID GLEE FINAL PERFORMANCE EVALUATION REPORT | 8

EVALUATION METHODOLOGY OVERVIEW

The final performance evaluation of the USAID GLEE project explored fifteen research questions across five key domains—relevance, effectiveness, efficiency, impact, and sustainability—along with five additional learning questions to inform future programming. These questions are outlined in the Answering the Evaluation Questions (EQs) section.

The evaluation also provided final values for seven of the USAID GLEE project’s indicators:

1. Girls who say they missed school because they feared school-related gender-based violence

(SRGBV)

2. Audience who recall hearing or seeing a specific USG-supported family planning (FP) or RH message

3. Girls with access to proper menstrual hygiene products

4. Schools that provide hygiene lessons for all students

5. Schools that provide orientation sessions on menstrual hygiene

6. Schools that have handwashing facilities

7. Improvement in girls’ menstrual hygiene knowledge following hygiene lessons

The mixed methods evaluation engaged a range of beneficiaries and stakeholders, including adolescent girls, school directors, teachers, USAID GLEE project staff, community members, and health personnel.

In May 2023, evaluation teams surveyed 75 school directors and 1,119 girls across USAID GLEE-supported primary schools, secondary schools, and ASCs in Bandiagara and Douentza using the beneficiary-based survey (BBS). Teams collected qualitative data from ten communities—five in Kayes, where the USAID GLEE project ended interventions in 2021, and five across Bandiagara and Douentza, where the project is still active. A total of 14 key informant interviews (KIIs) and 26 focus group discussions (FGDs) were conducted, and insights were triangulated with quantitative findings.

ANSWERING THE EVALUATION QUESTIONS

RELEVANCE

EQ 1: How relevant have USAID GLEE’s objectives, priority interventions, and approach been to the situation of the beneficiaries?

Answer to EQ 1: Nearly all KII and FGD respondents said that USAID GLEE’s activities appropriately addressed the needs of adolescent girls in their communities, with responses citing all of the areas of need that USAID GLEE targeted with its objectives. Activities that addressed specific project objectives included those related to girls’ access to education; their safety and security; their socioemotional health and sexual and reproductive health; their own economic insecurity, as well as that of their families and schools; and the prevalence of child marriage.

EQ 2: How has the original design evolved during USAID GLEE’s implementation, particularly in response to the findings from the midterm study?

Answer to EQ 2: During the project's life, USAID GLEE project staff and partners demonstrated their ability to change the design of interventions as needed to increase their effectiveness and impact.

Notably, they responded to a recommendation in midterm research by redesigning the activity to provide sanitary pads to girls and improve the monitoring of transferred learners from ASCs to primary and secondary schools.

EQ 3: How were existing relevant USAID and U.S. government activities leveraged?

9 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

Answer to EQ 3: USAID GLEE successfully created a constructive collaboration with other USAID-funded projects in the regions of Bandiagara and Douentza. For instance, they combined forces with a USAID agricultural project Sugu Yiriwa to improve women’s livelihoods by training women on various income-generating activities so that they could support their children's education with the resources they generated.

EFFECTIVENESS

EQ 4: To what extent has the project achieved its objectives as defined in the project’s results framework and reporting indicators?

Answer to EQ4 (by Objective):

OBJECTIVE 1: DECREASE KEY BARRIERS TO ACCESS QUALITY EDUCATION

• USAID GLEE’s work to establish ASCs tapped into a deep-felt desire for out-of-school girls to receive an education, and the project’s payment of school fees increased enrollment at government schools.

• USAID GLEE provided teaching and learning materials to schools to meet critical shortages.

• Multiple teachers, school directors, and ASC facilitators discussed the teaching methods and approaches they acquired from USAID GLEE pedagogical training, including learning about the balanced approach to literacy instruction and methods for coping with girls who need socioemotional support.

• USAID GLEE designed activities to educate girls, their families, and the community at large about the importance of girls’ education. These activities were organized at all 75 sampled primary and secondary schools, according to school directors. In addition, 90.7 percent of girls reported participating in a session on the topic.

• Nearly all girls—96.1 percent—agreed or strongly agreed that girls have as much of a right as boys to attend school. However, girls’ opinions differed when specifically asked about whether girls and boys had the right to stay in school if they were married or had a child, with a lower proportion agreeing or strongly agreeing that girls had that right compared to boys if they were married (70.9 percent to 82.8 percent, respectively) or had a child (73.9 percent to 87.2 percent).

OBJECTIVE 2: IMPROVE ADOLESCENT GIRLS' SAFETY IN SCHOOLS AND COMMUNITIES

• According to results from the quantitative survey, the majority of girls agreed or strongly agreed that both girls and boys were safe at school—92.2 percent and 93.3 percent, respectively.

• In KIIs and FGDs, nearly all respondents said they believed schools were safer thanks to USAID GLEE. Respondents cited the emergency plans they had adopted and the awareness sessions on safety and security conducted with parents, teachers, and students.

• Slightly more than one-quarter of girls said they knew someone in their class who used their school’s incident box, which USAID GLEE provided at each school so students could anonymously report cases of SRGBV or submit other school-related complaints.

• When asked about students’ use of incident boxes in KIIs and FGDs, responses were as mixed as girls’ survey responses. At some schools, respondents said that students did not use the box at all or used it infrequently. At schools where respondents said students did use the incident box, the submitted complaints more often concerned school-related issues unrelated to SRGBV.

• A greater proportion of girls in secondary school who were 13 or older agreed or strongly agreed with the statement that it was sometimes a girl’s fault if a teacher sexually harassed her

USAID.GOV USAID GLEE FINAL PERFORMANCE EVALUATION REPORT | 10

(65.4 percent) or a student did so (65.6 percent) than did girls 13 or older in primary school—

44.1 percent and 45.4 percent.

• The proportion of girls who agreed or strongly agreed that teachers touch children’s thighs, behinds, or private parts at their school varied by school type, with 7.5 percent of girls in primary schools and 25.5 percent of girls in secondary schools agreeing or strongly agreeing that teachers did so.

• In primary school, 8.4 percent of girls aged 13 or older agreed or strongly agreed that teachers demanded sexual relations, compared with 18.8 percent of secondary-school girls.

• In the quantitative survey, 11.2 percent of primary school girls and 6.2 percent of secondary school girls reported missing at least one day of school due to feeling unsafe on the way to, at, or returning from school. The difference between the proportion of girls in primary and secondary school who said they missed at least one day of school, however, was not statistically significant.

OBJECTIVE 3: INCREASE KNOWLEDGE AND ADOPTION OF POSITIVE HEALTH BEHAVIORS

AMONG ADOLESCENT GIRLS

• A majority of girls overall (71.0 percent) said they knew where to go if they needed information about menstruation.

• Slightly more than half of the girls who said they had started their period also said they were comfortable going to school while menstruating. A lower proportion—11.6 percent—of girls said they had been absent from school at least once due to their period. Overall, more than one-quarter of girls (26.3 percent) said they had started menstruating, with the highest proportion, by far, made up of secondary-school students (72.4 percent).

• Girls who said they had started their period had differing views about it. For instance, 46.3 percent of girls agreed or strongly agreed that they were ashamed of their bodies when they had their period, but 70.1 percent agreed or strongly agreed that they were proud of having their period. Some girls responded with views about their period that seem contradictory. Of the girls who somewhat agreed or strongly agreed that they were proud of having their period,

34.6 percent—more than one-third—somewhat agreed or strongly agreed that they were ashamed of their body when they had their period.

• Slightly more girls reported using cotton (34.8 percent) or pieces of fabric (33.9 percent) than sanitary pads (30.8 percent) to manage their period.1 Most girls (86.1 percent) reported always having access to at least one of these materials during the past three months.

• Girls and school directors diverged on responses to how often sessions on assorted topics occurred at schools. For instance, while 67.9 percent of girls reported that they had seen or heard about FP and RH during the current school year, nearly all school directors reported that their school had held sessions on FP and RH—95.4 percent and 97.2 percent, respectively.

• As for menstrual hygiene sessions, 93.0 percent of school directors said they had held sessions on the topic, with 43.8 percent of school directors reporting the most recent session had occurred in the past month.

• As part of the quantitative survey, girls were asked various questions to gauge their knowledge about menstruation. Overall, girls answered 25.3 percent of questions correctly about the topic, 1 Although USAID GLEE focused its activities on girls making their own sanitary pads from locally available materials, it was not clear from the quantitative survey how girls had acquired their sanitary pads.

11 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

but the difference was statistically significant between girls enrolled in ASCs (12.3 percent), primary schools (22.0 percent), and secondary schools (49.9 percent).

• The knowledge gap was present between girls by school level and if a girl said she had started her period. For instance, the first survey item inquired if the girl knew what a period or menses

is. Among secondary school students, 86.7 percent reported knowing what a period is, while

37.1 percent of ASC students and 53.5 percent of primary school students reported knowing what a period is.

• Nearly all of the school directors interviewed—95.1 percent—reported that their school provided lessons on general hygiene to students.

• As part of the school director survey, enumerators also took an inventory of schools’ handwashing facilities. Most schools—71.7 percent—had handwashing facilities, with 63.7 percent being within 10 meters of the latrines. As for branding, 33.9 percent of the facilities had USAID branding.

• The availability of water and soap or ashes varied at the handwashing facilities. Water was available at all handwashing facilities at 32.8 percent of schools, in more than half of facilities at

25.0 percent of schools, and in less than half of facilities at 25.5 percent of schools. At 16.6 percent of schools, water was not available at any handwashing facilities. More than one-quarter of schools had soap or ashes available at all handwashing facilities, but they were not available in

40.8 percent of schools.

EQ 5: What were the major factors—including project design, implementation, and the operating environment—that influenced the achievement or non-achievement of the objective targets? and EQ 6:

Which project activities made the most and least significant contribution to intended strategic objectives?

Answer to EQs 5 and 6: USAID GLEE’s dedication to engaging with communities from the moment its staff first arrived stands out as the most critical factor in achieving project objectives. With a project design relying heavily on community participation—including its initial community assets appraisal approach plus community members fulfilling key roles such as mentors and ASC facilitators—USAID GLEE needed to build local ownership and trust to be able to make an impact. Based on KIIs with USAID GLEE project staff and partners, that paramount objective was met.

EQ 7: How do USAID GLEE beneficiaries perceive the overall quality of project delivery and technical assistance?

Answer to EQ 7: Respondents generally viewed the project’s overall activities and assistance as effective. When asked which USAID GLEE activities were the least effective, most respondents in KIIs and FGDs said they believed all activities were effective. The respondents who said certain activities were ineffective were not critical of the activities’ quality or design, but rather wished USAID GLEE had done more to support the community.

EQ 8: How did USAID GLEE adapt to the pandemic and to what extent were adaptations or shifts in the program’s delivery strategy required to reach USAID GLEE’s beneficiaries? How do USAID GLEE beneficiaries perceive the quality of the program’s adaptation?

Answer to EQ 8: The pandemic forced USAID GLEE to adapt its school-based activities when government schools closed at the onset of the pandemic, ramp up some community-based interventions, and distribute materials to communities to prevent transmission of the Coronavirus Disease 2019 (COVID-19). While beneficiaries shared their appreciation for the project's support in response to the pandemic, most respondents believed that COVID-19 had little to no overall impact in

USAID.GOV USAID GLEE FINAL PERFORMANCE EVALUATION REPORT | 12

their communities, with some stating that they believed there had been no COVID-19 cases in their community.

IMPACT

EQ 9: What were unexpected outcomes of USAID GLEE activities, both positive and negative outcomes, particularly for girls?

Answer to EQ 9: As part of the qualitative data collection, 71 adolescent girls who benefited from USAID GLEE participated in FGDs designed to prompt each girl to share a story detailing the most significant change (MSC) in her life that resulted from the project and then, as a group, select the story that captured the MSC out of the stories shared. Out of the 12 FGDs conducted with girls, five groups selected a story related to early marriage (see page 57 for the story details). The stories related to child marriage illustrate how USAID GLEE has raised awareness of various issues related to girls’ lives and empowered girls to change the course of their own lives and impact the lives of other girls in their communities. In the FGDs, girls shared notable stories of empowerment and agency related to RH and FP. These stories illustrated how girls used their newfound knowledge to push back against prevailing social norms—especially how some people feel FP and RH are inappropriate for adolescent girls to learn about—as well as girls’ determination and desire for seeking out FP and other health services, which surprised even healthcare workers themselves.

EQ 10: What changes in the enabling environment that support girls’ education and school safety have resulted from USAID GLEE?

Answer to EQ 10: Where USAID GLEE intervened, its collaborative approach helped to raise awareness about the importance of educating girls and creating a safe school environment. Respondents shared how all these efforts led not only to behavioral change in families, such as allowing girls to study instead of performing household duties, but also shifted deep-rooted community beliefs that girls did not belong in school. Other respondents described how girls manifested their newfound independence, with some refusing to have their parents force them to marry young. As for school safety, respondents primarily described how some communities ensured all children arrived at school and returned home safely and that the school grounds were adequately fenced off and secured.

EFFICIENCY

EQ 11: To what extent does the management structure support efficiency for implementation, learning, and reflection for WI and partners and ensure proper risk management?

Answer to EQ 11: With a focus on objectives in three sectors—education, health, and safety and security—USAID GLEE required a complex partnership of international and local organizations to implement activities. All project and partner respondents said that the consortium operated effectively, with one respondent recommending that the consortium could have been strengthened with more personnel from WI working out of the regional office in Bandiagara. The consortium’s quarterly meetings played a key role in efficient project management. The project’s management structure also helped it ensure proper risk management, with local partners having access to key information that allowed the consortium to assess the security situation in Douentza and Bandiagara.

EQ 12: Did any activities with relatively high impact or effectiveness have higher reach of beneficiaries than others?

Answer to EQ 12: The quantitative survey and girls’ FGDs revealed that certain activities had more reach with girls. In the survey, girls reported that USAID GLEE mentors benefited them the most (72.6

13 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

percent of girls overall), followed by the production of sanitary pads (47.6 percent overall). In the stories of MSC that girls shared in FGDs, early marriage was the topic of the stories selected most often by groups, while girls also shared stories frequently about learning about RH and FP and how to produce reusable sanitary pads; becoming more aware of the importance of staying in school and receiving an education; and benefiting from USAID GLEE’s provision of school fees, water, school supplies, bicycles, brooms, and benches.

SUSTAINABILITY

EQ 13: What is the likelihood that the project benefits will endure over time after USAID GLEE ends?

Answer to EQ 13: Due to USAID GLEE ending its work in Kayes in 2021, it was possible to see which project activities were still being conducted there as a real-time gauge of sustainability. The activities with the most staying power in Kayes include the ongoing awareness raised in communities by mentors, peer educators, and youth ambassadors. Other activities that have continued include the care provided by health agents to adolescent girls (i.e., family planning and routine visits for illness) and the production of reusable sanitary pads. However, two challenges were found that affected USAID GLEE’s prospects for sustainability—the deterioration of infrastructure, such as WASH facilities, and the departure of trained mentors, teachers, and health workers from communities.

EQ 14: To what extent has USAID GLEE developed local ownership and sustainable partnerships?

Which, if any, improved institutions or processes are likely to continue after completion of USAID

GLEE?

Answer to EQ 14: As detailed in EQ13, project activities—such as the local production of sanitary pads—are likely to continue because the community has taken ownership of the practice. Community health centers also appear to have institutionalized services for adolescent girls. By contrast, in the education sector, some GoM officials in Kayes discussed how the progress USAID GLEE had brought about resulted in some unforeseen challenges for the government once the project closed, primarily officials feeling obligated to open and operate new schools in communities where ASCs had closed despite “meager” financial resources.

CONCLUSIONS AND RECOMMENDATIONS

USAID GLEE’s impact expanded beyond the walls of classrooms into all facets of girls’ lives, as exemplified by their stories of MSC about early marriage, RH, and FP. In these stories, girls not only recounted how the project had helped change the course of their own lives, but also how the newfound knowledge empowered them to make a difference in the lives of their peers.

Recommendation: USAID GLEE’s design of its reusable sanitary pad production should be highlighted as a best practice. Due to its popularity and sustainability prospects, the project’s production of sanitary pads with local materials should be incorporated into other projects.

Girls not only changed their view of what they could accomplish in their own lives, thanks to project activities, but what they accomplished over the life of project also changed community perceptions of what girls were capable of achieving. These changed perceptions underscore how adolescent girls’ potential in Mali is generally untapped. Through its active presence in communities, USAID GLEE convinced people otherwise and taught them to be more attentive to girls.

Recommendation: Communities’ surprise at what girls are capable of achieving illustrates how it is important not only to raise awareness in communities about the intrinsic importance of girls’ education, but also about what girls themselves are capable of accomplishing. This

USAID.GOV USAID GLEE FINAL PERFORMANCE EVALUATION REPORT | 14

realization points to the type of shift in gender norms that effective empowerment approaches help bring about. Gender empowerment models need to attend to the individual-level change in opportunities and self-concept for girls, but also to the peers, families, and communities that make up the enabling environment around girls. USAID GLEE’s mix of interventions show a sound and practical direction in programming and implementation for gender empowerment at these multiple levels of the enabling environment and should be continued.

Despite the impact the project made in girls’ lives, USAID GLEE was less successful in shifting deep-seated cultural attitudes about gender in society and SRGBV. For instance, while a minority of primary school girls 13 and older agreed or strongly agreed with the statement that it was a sometimes a girl’s fault if a teacher or student sexually harassed her—44.1 percent and 45.4 percent, respectively—a majority of secondary school girls did—65.4 percent and 65.6 percent, respectively.

Recommendation: Activities targeting SRGBV awareness need to address problematic gender norms that lead girls to believe they are sometimes to blame for provoking harassment. While the above recommendation points out the evidence of effectiveness and potential in shifting gender norms, this conclusion and recommendation indicates that progress within girls’ self-concept and in their placement of blame on themselves rather than the adults harassing them can be slow and difficult. In future programming, this specific area of social-emotional learning for adolescent girls can be more specifically targeted and peers, family members, and community members can also be enlisted in this area of social-emotional growth.

It is evident that boys and men need to change their behavior to ensure schools are safer and that the criteria that children consider when determining their school’s safety should include instances of SRGBV. For example, while 93.4 percent of secondary school girls said girls were safe at school, 25.5 percent said that teachers touched children inappropriately, and 18.8 percent said that teachers demanded to have sex with certain students.

Recommendation: Boys and men need to be targeted explicitly in the design of activities addressing SRGBV. As with the above, this recommendation is another call to take up further emphasis in this related area of the enabling environment among the peers and adults whose behaviors have a large effect on girls’ emerging sense of self, locus of control, and opportunities for their future.

As for menstrual health, although respondents noted how the project helped to improve girls’ menstrual health hygiene and knowledge, the stigma associated with girls having their period still manifested itself in some responses in the girls’ survey. Despite the lingering stigma, the project made an impact with menstrual health and hygiene by increasing access to sanitary pads with its well-received activity of using locally available material to produce them. Mentoring in communities stood out as another well-received activity with promising prospects for sustainability. In the quantitative survey, girls reported that USAID GLEE mentors benefited them the most out of any project activity.

Recommendation: The use of mentors should be expanded so that multiple mentors are active in each community. This is another promising practice for a holistic gender empowerment approach that builds community understanding, support, and structures to encourage and build the enabling environment.

The reach of the awareness-building sessions that mentors and others were trained to lead at least weekly, however, seemed unclear. For instance, although nearly all school directors reported that their schools conducted sessions on RH as well as FP, less than 70 percent of girls reported having

15 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

participated in such sessions or seeing or hearing messages related to those topics. Perhaps girls who said they did not participate in these may not have been present at school when mentors and others facilitated the weekly session.

Recommendation: Due to a sizeable minority of girls reporting they did not participate in sessions on such topics as menstrual hygiene, FP, and RH, girls’ participation and exposure to these sessions should be monitored more closely, as well as their ongoing learning, including application of their learning to their lives. Later in the school year, mentors and other project stakeholders should be encouraged to repeat trainings on certain key topics if monitoring reveals that a large proportion of girls have not attended certain sessions, or their learning should be reinforced on a certain important topic such as menstrual health.

In Kayes, the sustainability successes included not only sanitary pad production and continued contributions from mentors and others, but also the continued provision of health services to girls at community health centers. Several challenges in Kayes, however, emerged with sustainability— infrastructure, such as water taps no longer functioning and stakeholders leaving the community.

Recommendation: The ability of community health centers in Kayes to continue covering the costs of adolescent girls should be studied more closely to see if aspects could be emulated to cover the costs of school fees.

Recommendation: To ensure infrastructure does not break easily, future projects should work with local government and school organizations on how to finance repairs and maintenance, especially with WASH facilities.

Recommendation: To mitigate the effects of the inevitable departure and turnover of project stakeholders such as mentors and teachers, future projects should ensure that protocols are in place to train replacements to take over duties as needed.

USAID.GOV USAID GLEE FINAL PERFORMANCE EVALUATION REPORT | 16

INTRODUCTION

CONTEXT AND PROJECT BACKGROUND

The Girls Leadership and Empowerment through Education (USAID GLEE) project is a five-year, USAID-funded project to increase girls’ educational opportunities in Mali by supporting and leveraging existing efforts by the Government of Mali (GoM) and civil society. The project is implemented in an unstable security context in which children and adolescents face myriad barriers to accessing quality education. Only 49 percent of learners in Mali currently complete primary school, driven to dropout by social and economic pressures borne disproportionately by girls.2 Expectations of domestic labor and care work, along with high rates of child marriage, place competing pressures on girls’ ability to enroll, attend, and complete schooling throughout their educational careers. Once at school, student learning outcomes are constrained by a lack of materials, the absence of qualified teachers, and the risk of gender-based violence (GBV) on the way to, at, and returning from school.

The USAID GLEE project developed a comprehensive approach to addressing girls’ educational challenges in Mali by focusing on decreasing barriers to accessing quality education, improving girls’ safety, and increasing knowledge and adoption of positive health behaviors. Winrock International (WI) leads the USAID GLEE consortium of four partners— Groupe d’Actions et d’Animation au Sahel–Mali (GAAS), IntraHealth International, and Œuvre Malienne d’Aide à l’Enfance du Sahel (OMAES) and Centre d’assistance et de promotion des Droits Humains en Afrique (CPHDA). GAAS and CPHDA implement community engagement and school support activities. OMAES is responsible for the accelerated schooling centers (centre de scolarisation accélérée, ASC) and teacher professional development activities. IntraHealth International implements reproductive health (RH) and water, sanitation, and hygiene (WASH) activities.

The USAID GLEE project launched in Mali in 2018 and, after five years of implementation, concludes in 2023. Throughout the project, WI used monitoring and evaluation touchpoints, including formal assessments, to better understand the population the project served. The 2018 USAID GLEE baseline evaluation included a household survey of girls and their parents to understand challenges within the target regions of Kayes and Mopti—the latter now split into the Bandiagara and Douentza regions.

Results showed that school access remained out of reach for a substantial proportion of girls, reflecting the broader trends across Mali. Only about one-half of the surveyed girls were currently enrolled in school. Of those enrolled in school, survey data indicated a large majority struggled to attend class regularly; three in four girls missed more than five school days in the previous month. It is likely that these prominent levels of absenteeism negatively affected girls’ ability to engage meaningfully in their studies and progress academically, increasing their vulnerability to dropout.

These challenges were particularly acute for girls aged 15 to 18 when compared with their younger peers. School survey data suggested that despite low overall dropout rates for girls in primary school (first grade through ninth grade), girls became more vulnerable to dropping out as they transitioned from primary to secondary school. This trend was not surprising given that, as girls age, they face increased domestic responsibilities, pressure to contribute economically to their household, sexual harassment, and cultural expectations of early marriage. Adolescent girls with menstrual health management needs also find additional challenges at school not faced by younger girls.

2 Additional information about education in Mali can be found at https://www.usaid.gov/mali/fact-sheet/girls-leadership-and-empowerment-through-education-USAID GLEE https://www.usaid.gov/mali/fact-sheet/girls-leadership-and-empowerment-through-education-USAID%20GLEE https://www.usaid.gov/mali/fact-sheet/girls-leadership-and-empowerment-through-education-USAID%20GLEE

17 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

In 2021, USAID GLEE’s mid-project research study further explored adolescent girls’ barriers in Mali through direct engagement with project beneficiaries. Respondents reported numerous obstacles, including household duties and obligations, financial demands, early marriage and pregnancy, parents’ lack of awareness of education’s importance, school-related gender-based violence (SRGBV), and health-related barriers, including menstruation. Meanwhile, a separate external midterm evaluation in 2021 found that the USAID GLEE project successfully worked with school management committees (comités de gestion scolaires, CGSs) by building the capacity of key community partners. This included community mobilization through mentors, peer educators, youth ambassadors, and grandmothers.

USAID GLEE had also successfully collaborated with the education, health, and community sectors on the project, although institutionalizing the partnerships was noted as a critical need for sustainability.

THEORY OF CHANGE AND USAID GLEE STRATEGY

Building upon other programs and efforts by the GoM, USAID GLEE’s overarching goal was to increase access to education for adolescent girls aged 10 to 18 and enable them to obtain greater educational attainment. The USAID GLEE project has three key objectives, each supported by multiple sub-results (SRs):

1. Objective 1: Decrease key barriers of adolescent girls to access quality education.

a. SR 1.1: Learning-support opportunities for adolescent girls increased

b. SR 1.2: Teaching is more responsive and relevant for adolescent girls

c. SR 1.3: Communities and families support girls’ education

2. Objective 2: Improve the safety of adolescent girls in schools and their communities.

a. SR 2.1: Strategies and policies to safeguard adolescent girls are adopted

b. SR 2.2: Effective reporting and referral mechanisms for SRGBV established

c. SR 2.3: Perceptions of safety in communities and schools improved

3. Objective 3: Increase knowledge and adoption of positive health behaviors among adolescent girls.

a. SR 3.1: Positive health behaviors adopted by adolescent girls

b. SR 3.2: Adolescent girls’ knowledge of and access to health services increased

c. SR 3.3: Health barriers to education reduced

USAID GLEE relied on seven strategies to meet these objectives:

1. Community mobilization through CGSs, parents’ groups (association des parents d’élèves, APEs), and other key community stakeholders.

2. Participatory capacity-building of ASC facilitators and teachers at partner schools.

3. Mentorship and peer learning through USAID GLEE mentors, youth ambassadors, and grandmothers.

4. Targeted social behavior change communications around girls’ education, safety, and health.

5. Payment of girls’ scholarships.

6. Provision of teaching and learning materials.

7. Improving WASH facilities at schools.

EVALUATION PURPOSE AND AUDIENCE

EdIntersect, with partners School-to-School International (STS) and Centre d’Etude et de Recherche sur l’Information en Population et Santé (CERIPS), conducted a mixed-methods final performance evaluation

USAID.GOV USAID GLEE FINAL PERFORMANCE EVALUATION REPORT | 18

near the close of the USAID GLEE project.3 The purpose of the final performance evaluation is threefold:

1. Assess the project’s achievements as outlined in the results framework.

2. Assess the project’s relevance, effectiveness, efficiency, impact, and sustainability.

3. Listen to and engage with girls as key informants on USAID GLEE’s outcomes.

This final performance evaluation follows three other USAID GLEE evaluations, as detailed in Table 1, including a baseline, an internal mid-project evaluation, and an external mid-project evaluation. Any baseline data mentioned in this report is only for referential purposes. No comparisons between baseline and the final performance evaluation can be made due to differences in the baseline and final performance evaluation designs.4

Table 1. Life-of-Project Data Collection Timeline, Sampling Approach, and Instruments Used

Data collection

Baseline Internal Mid-Project

External Mid-Project

Final Performance Evaluation

November 2018 November– December 2020

April 2021 May 2023

Sampling approach

Random route sampling method was used to visit households in Kayes, Bandiagara, and Douentza; both girls attending and not attending school were surveyed; all accessible schools targeted for participation in USAID GLEE were visited

10 USAID GLEE

communities selected for qualitative data collection in Kayes, Bandiagara, and Douentza

Population-based sample drawn from entire population of

USAID GLEE

schools in Kayes, Bandiagara, and Douentza as well as non-project control schools from the same regions

Population-based sample drawn from entire population of USAID GLEE schools in Bandiagara and Douentza for quantitative instruments; no control schools;

qualitative sampling including a total of 10 communities, 5 in Bandiagara and Douentza, and 5 in Kayes

Instruments Household girls’ survey and parent/caregiver survey, as well as a school survey

No quantitative instruments; seven FGD and KII protocols

Girls’ survey, school inventory and observation checklist, and school director survey; nine FGD and KII protocols

Beneficiary-based surveys for girls and school directors;

11 FGD and KII protocols for range of beneficiaries

SSME data collection

The final performance evaluation results will be shared with USAID, project staff, partner organizations, and other key stakeholders in Mali. The performance evaluation reports on seven of USAID GLEE’s thirty-two key performance indicators included in the Activity Monitoring, Evaluation, and Learning Plan

3 This is considered an internal evaluation as it is commissioned by Winrock International, the lead USAID GLEE implementer.

4 While the final performance evaluation sample was drawn from girls who attended schools where USAID GLEE has been intervening, the baseline sample was drawn from school-aged girls who lived in households in communities where USAID GLEE was planning to intervene. In addition, unlike final performance evaluation data, baseline data was unweighted.

19 | USAID GLEE FINAL PERFORMANCE EVALUATION REPORT USAID.GOV

(AMELP). In addition to collecting primary data on these seven indicators, the evaluation team conducted a desk review of WI’s project monitoring…