ATTACHMENT_J.8_Endline_Report.pdf

EdData II

Measurement and Research Support to Education Strategy Goal 1 Malawi Social and Behavior Change Communications Pilot: Endline Report

Education Data for Decision Making (EdData II) Period of Performance: 10/1/2012 – 11/30/2016 Task Order Number AID-OAA-BC-12-00003 RTI Project No. 0209354.020 (Task 20, Activity 11)

October 2016

This publication was produced for review by the United States Agency for International Development. It was prepared by RTI International.

Measurement and Research Support to Education Strategy Goal 1 Malawi Social and Behavior Change Communications Pilot: Endline Report

Prepared for:

Office of Education Bureau for Economic Growth, Education, and Environment (E3) United States Agency for International Development (USAID) Arthur Muchajer, Contracting Officer Penelope Bender, Contracting Officer’s Representative

Prepared by:

Karen Schmidt, Joseph DeStefano, and Stirling Cummings RTI International 3040 East Cornwallis Road Post Office Box 12194 Research Triangle Park, NC 27709-2194

The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.

RTI International is a registered trademark and a trade name of Research Triangle Institute.

Malawi SBCC Endline Report iii

Table of Contents

List of Figures ............................................................................................................................ iv

List of Tables ............................................................................................................................. iv

Abbreviations .............................................................................................................................. v

Executive Summary

1 Introduction

2 Methods

2.1 Sampling

2.2 Analytical Approach

3 Household Survey Results

3.1 Campaign Recall

3.1.1 Unprompted Recall

3.1.2 Prompted Recall

3.1.3 Campaign Components and Process Indicators

3.2 Attitudes, Norms and Self-Efficacy

3.2.1 Attitudes

3.2.2 Norms

3.2.3 Self-Efficacy

3.3 Behavior

4 Results of Teacher Survey

4.1 Teacher Attitudes

4.2 Classroom Activities Promoting Reading and Availability of Reading Materials

5 Discussion

6 Limitations

References

Malawi SBCC Endline Report iv

List of Figures

Figure 1. Ntcheu District, Malawi

Figure 2. Integrative Model of Behavior Change

Figure 3. Kasinje: Attitudes – Role parents can play to help their child with school (unprompted)

Figure 4. Kasinje: Norms

Figure 5. Kasinje: Self-efficacy

Figure 6. Kasinje: Parent interaction with child about school

Figure 7. Kasinje: Behavior – How household members help the child with school, pre vs. post

List of Tables

Table 1. Unprompted and prompted recall about campaign media (percentages)

Table 2. Percentage of “yes” responses, prompted and unprompted recall about specific messages

Table 3. Kasinje: Attitudes – Role parents can play to help their child with school (unprompted) (percentages)

Table 4. Kasinje: Norms, pre vs. post (percentages)

Table 5. Kasinje: Norms by exposure (percentages)

Table 6. Kasinje: Self-efficacy, pre vs. post (percentages)

Table 7. Kasinje: Self-efficacy by exposure (percentages)

Table 8. Kasinje: Behavior – How household members help the child with school, by exposure (percentages)

Table 9. Likelihood of reporting certain behaviors as a result of the campaign

Table 10. Teachers’ perceptions of roles parents can play to help their child with school (percentages)

Table 11. Frequency of parents’ visits to schools (percentages)

Table 12. Discussion topics when parents visited schools (percentages)

Table 13. Frequency with which teachers assign homework (percentages)

Table 14. Children taking school books home (percentages)

Table 15. Children taking story books or story cards home (percentages)

Malawi SBCC Endline Report v

Abbreviations

E3 USAID Bureau for Economic Growth, Education, and Environment EdData II Education Data for Decision Making IKI Invest in Knowledge Initiative (research organization, Malawi) IRT item response theory OR odds ratio RTI RTI International (a registered trademark and trade name of Research

Triangle Institute) SBCC social and behavior change communication USAID United States Agency for International Development

Malawi SBCC Endline Report 1

Executive Summary To reinforce school-based efforts to improve early grade reading, ministries of education and their technical and financial partners are paying increased attention to how families can help children build literacy skills at home. This report describes an activity in Ntcheu District in Malawi, sponsored by the United States Agency for International Development (USAID), which was designed to use social and behavior change communication (SBCC) techniques to mobilize parents to help their children learn to read. The SBCC campaign was evaluated with a household survey, with a baseline survey being conducted in March 2016 and the endline survey in September 2016 after the intervention. The survey explored the knowledge, attitudes, beliefs, and behaviors of family members of students enrolled in Standards (grades) 1–3. In addition, the teachers at the schools from which students were sampled were interviewed.

Invest in Knowledge Initiative (IKI), a Malawian contract research organization, collaborated with the USAID-funded Early Grade Reading Activity to select a sample of households of students enrolled in two zones: Kasinje, which served as the intervention area; and Senzani, which served as the comparison area. In addition, a separate survey queried the teachers of the classrooms from which students/households were sampled.

Parents. The survey showed that, despite some implementation challenges, campaign recall was at acceptable levels, albeit much lower than the levels of recall seen in both areas where SBCC activities were piloted in Senegal. When the respondents were asked to report if they had seen or heard any messages about children learning to read in the previous few months (unprompted recall), 41 percent said yes. When the interviewer described the campaign specifically, and then showed the logo, the recall rose to 68 percent overall (prompted recall).

When it came to parents’ attitudes, norms, and self-efficacy, as well as their behavior, the most notable and statistically significant differences were seen when low-exposure Kasinje respondents were compared with high-exposure ones. When asked about a parent’s role in their child’s reading and schooling, a high-exposure Kasinje respondent was twice as likely to cite “have the child read aloud, “read with the child,” and/or “check the child’s notebook.”

Norms, as measured by the question “Do you know of friends and neighbors who read with their children?” showed significant changes pre- and post-campaign in Kasinje, and when the results were analyzed by exposure, the odds of a high-exposure Kasinje respondent saying “yes” were almost twice those of a low-exposure Kasinje resident. Self-efficacy was fairly high at baseline:

When asked, “Do you feel you can help your child learn to read?” 70 percent in Kasinje and 77 percent in Senzani said yes. The proportion was higher post-campaign in Kasinje as compared to post-campaign. Exposure analysis showed that the odds of a high-exposure Kasinje respondent saying “yes” were almost four times that of a low-exposure Kasinje resident.

The same pattern was seen when respondents were asked if they helped their child with school work: about two-thirds said yes at baseline and endline; however, at endline, the odds of a high-exposure Kasinje respondent saying “yes” were nearly four times those of a low-exposure respondent. When those who said they helped were asked how they helped, the high-exposure

Malawi SBCC Endline Report 2

Kasinje residents were significantly more likely to cite behaviors promoted by the campaign, such as “read to child,” “ask child to show me his/her work,” and “ask child to read to me.” When respondents were asked if anyone in the household helped the child with school work, exposure analysis showed that 39 percent of high-exposure Kasinje residents cited “Practices reading with child” compared to 25 percent of low-exposure residents. The proportions for other behaviors were lower, but the differences were mostly statistically significant.

When respondents were asked if the campaign made them more likely to read with their child at home, 56 percent said it had “a lot of impact” and 10 percent said it had “a little impact.” The proportion was similar for “do educational activities with your child at home.” Sixty-seven percent said the campaign made them more likely to talk to their child about school, and 51 percent said it made them more likely to encourage other parents to read or do other educational activities with their children.

Teachers. Results of the teacher survey showed a less consistent pattern, and relatively few results were statistically significant. When teachers were asked what role they believed parents could play in helping their child with school, three key behaviors promoted by the campaign— “read with the child,” “check child’s homework,” and “do homework with the child”—showed significant increases from baseline to endline in Kasinje. Teachers were asked how often parents visited the school, and what they wanted to discuss when they visited. In Kasinje, the only significant change from baseline to endline was in the proportion of parents who wanted to discuss learning achievement; this figure doubled from 20 percent at baseline to 40 percent at endline.

Overall, these results suggest that SBCC is a promising intervention for increasing parental involvement in children’s reading and schoolwork. The USAID Malawi Early Grade Reading Improvement Activity (MERIT, 2015–2020) offers opportunities to further refine the intervention and determine how different packages of components might perform.

Malawi SBCC Endline Report 3

1 Introduction With funding from the United States Agency for International Development’s (USAID’s) Office for Economic Growth, Education, and the Environment (E3), the Education Data for Decision Making (EdData II) project has investigated how to employ social and behavior change communication (SBCC) strategies to promote and stimulate home-based support for children learning to read. Recognizing that the dramatic improvements needed in reading outcomes in most developing countries will likely be achieved only through a combination of in-school and at-home efforts in support of early literacy, USAID wanted to test how the lessons from successful SBCC campaigns in the health sector could be applied in education. Following an initial pilot SBCC campaign in one district in Senegal (see RTI International, 2015), EdData II initiated a second round of research—in a second district in Senegal (RTI International, 2016), and in one district in Malawi (DeStefano & Cummings, 2015).

The SBCC activity in Malawi was designed in a fashion similar to the one implemented in Kaolack, Senegal,1 making use of a multichannel approach to reach families in an area where an early grade literacy program was already being implemented. The multichannel campaign involved delivering messages to parents through radio spots, a radio program, and posters hung in prominent locations within each community. The campaign also included reinforcement of those messages through community meetings, radio listening groups, and community theater performances that reiterated the importance of reading and demonstrated to parents specific activities they could do at home to support their children learning to read.

The USAID-funded Early Grade Reading Activity (June 2013–October 2016) was active in 11 districts in Malawi, one of which, Ntcheu, was selected for implementation of this pilot SBCC campaign. Because the multichannel approach was successful in Senegal at promoting positive changes in households’ attitudes, beliefs, and behaviors, the pilot in Malawi was set up to validate/replicate those findings in a different context. In collaboration with the Malawi Early Grade Reading Activity, EdData II implemented the SBCC campaign in the communities associated with 16 schools in the zone of Kasinje. Another group of communities in a different zone (Senzani) in Ntcheu District served as a comparison (no SBCC activities took place there).

In conjunction with the Malawi Early Grade Reading Activity staff and a local partner organization, Invest in Knowledge Initiative (IKI), a baseline survey was completed in March 2016 (DeStefano & Cummings, 2015) and an endline survey was conducted in September 2016. The results of this research are the subject of this report.

1 For more information about the Senegal SBCC campaign, see RTI International (2015, 2016).

Malawi SBCC Endline Report 4

2 Methods As noted above, the evaluation of the Malawi behavior change communications pilot intervention was conducted by comparing the intervention study area, Kasinje, and the comparison area, Senzani—two zones in the Ntcheu District (Figure 1). Pre-intervention and post-intervention data were collected from both areas through a structured questionnaire. The two areas are separated by a distance of approximately 70 km, which was judged to be a sufficient distance to avoid communication spillover.

2.1 Sampling

All 16 schools from the intervention zone and all 15 schools from the comparison zone were included in the study. From each school, 30 learners were randomly selected from Standards (grades) 1, 2, and 3. Each class contributed 10 learners, half of whom were boys and half girls. Once selected, a learner’s household was identified and the parent/guardian then interviewed.

The instruments used for the parent and teacher interviews were developed by RTI International and further adapted to fit the local context by IKI. IKI translated the questionnaire into Chichewa, trained the interviewers, and carried out the fieldwork. Data capture in the field was carried out using electronic tablets with CSPro2 software.

2 Developed by U.S. Census Bureau and ICF International. For more information, see:

http://www.census.gov/population/international/software/cspro

Figure 1. Ntcheu District, Malawi http://www.census.gov/population/international/software/cspro

Malawi SBCC Endline Report 5

2.2 Analytical Approach

To evaluate the impact of the SBCC campaign in Kasinje, we used statistical analyses suited to the observational and cross-sectional nature of the data. Three types of comparisons were conducted:

(1) a comparison of baseline and endline results in the intervention site, Kasinje;

(2) a comparison of endline results from the control site, Senzani, to endline results from Kasinje; and

(3) a comparison of endline results based on the level of exposure to the media campaign in Kasinje.

Comparisons (1) and (2) relied on propensity score matching, and then conditional logistic regression, in order to reduce bias that could result from an unbalanced sample of households.

In observational studies, such as the Malawi SBCC campaign, where the individual households could not be randomized into intervention or treatment groups, there is a potential for significant bias from confounding factors evident in individual respondents and the environment in the treatment and control zones. In an effort to counteract this effect, we used propensity score matching to match a pre-campaign household to a post-campaign household in the same zone, or an intervention household to a control household in different zones. This method attempted to reduce bias by pairing each intervention zone respondent to a comparison respondent based on potentially confounding factors. Factors used for the matching included age of the respondent, sex of the respondent, age of the child, grade of the child, educational attainment of the respondent, and household wealth score. After matching, we employed conditional logistic regression to test for differences, based on the concordance or discordance of the outcome, in the matched pairs. Results were reported as odds ratios (ORs) and the corresponding confidence intervals.

To compare responses based on campaign exposure, we created a measure of media exposure from questions about whether the respondents recalled general and specific information from the media campaign. The questions ranged from the campaign channels (radio, public performances, posters and printed materials, etc.) to specific questions about the campaign message and characters in the radio spots and public performances. Item response theory (IRT) was used to calculate the measure, or index, of what respondents recalled of the media campaign. The use of the resulting scores was based on the assumption that subjects’ responses were tied to a single latent trait, which was the exposure to the campaign message from multiple sources.

After the exposure score was calculated for each respondent, we again used logistic regression to test for associations between knowledge, attitude, and behavior outcomes in the household that promote literacy, and the exposure score, while controlling for respondent and household characteristics. In order to promote interpretability of the resulting odds ratios, we recoded the exposure variable resulting from the IRT analysis as either high exposure (upper 50th

Malawi SBCC Endline Report 6 percentile) or low exposure (lower 50th percentile). The analysis compared the odds of high-exposure respondents vs. odds of low-exposure respondents within the same zone and post-campaign time point.

3 Household Survey Results

3.1 Campaign Recall

3.1.1 Unprompted Recall

For the endline survey in September 2016, respondents in Kasinje (the intervention zone) were asked a series of questions to measure their awareness and recall of the campaign.

Respondents were first asked to report, spontaneously, if they had seen or heard any messages about children learning to read in the previous few months. Forty-one percent said they had seen or heard some messages about children learning to read. When those respondents were then asked where they had seen or heard the messages, 13 percent said radio; 9 percent said posters, banners, or signs; 8 percent cited community mobilizers; and 7 percent cited theater groups, all of which were components of the campaign. When asked how often they had seen or heard such advertising or information, 23 percent said every day and another 29 percent said one to three times per week. Unprompted and prompted recall of individual messages is presented in Table 1.

Table 1. Unprompted and prompted recall about campaign media (percentages)

Campaign medium Unprompted Prompted Overall 41 68

Radio 13 32

Radio spot n/a 6

Radio program n/a 20

Posters / banners / signs 9 59

Community theater 7 43

Community mobilizer / interpersonal communication 8

Attended community meetings / radio listening clubs 18 n/a = Not applicable. The interviewers asked respondents where they had heard or seen messages but did not ask them to distinguish between radio spots and the radio program.

Note: Column percentages add to more than 100 because of the option for respondents to supply more than one answer.

Malawi SBCC Endline Report 7

3.1.2 Prompted Recall

When the interviewer specifically prompted the respondents by describing the campaign and asking if they recalled it,3 49 percent said “yes.” When those who said “no” were shown the logo, another 19 percent said “yes,” for a total prompted recall rate of 68 percent (not shown).

Among them, 26 percent said they heard or saw something related to the campaign “sometimes” and 35 percent said “often” or “very often.” When asked about the sources of the messages, 31 percent said they had heard it on radio, 59 percent said they had seen posters, and 43 percent had seen drama performances.

When asked about specific messages, those that the respondents were most likely to recall were “the role of illiterate parents in helping their children read” and “the benefit of reading” Overall, posters were the channel through which respondents were most likely to recall specific messages (Table 2).

Table 2. Percentage of “yes” responses, prompted and unprompted recall about specific messages

3.1.3 Campaign Components and Process Indicators

The Tiwerenge (Let’s Read) campaign was designed at a creative workshop in Kasinje that included radio presenters and producers, teachers and other school officials, an artist, and members of a theater for development group, as well as community mobilizers from the Early

3 The interview question was as follows: “Recently in the community there was an advertisement/campaign about the importance of reading for children. The campaign described what parents can do to help their early grade children with reading/schoolwork. Do you recall this campaign?”

Question: “Did you recall this campaign message?”

Un-prompted

Prompted

Radio Posters Theater

The role of illiterate parents in helping their children read 14.6 11.8 22.0 16.9

The benefit of reading 10.9 10.0 14.9 13.5

Kuwerenga ndi maziko akuzindikira (Reading is the foundation of knowledge) 11.3 7.8 15.1 10.4

Mwana amene amatha kuwerenga amanyadiritsa makolo (A child who knows how to read brings pride to his/her parents)

8.2 4.9 12.2 5.1

Providing books and reading material to our children 3.1 3.1 6.4 4.2

Kukonza nthawi yapaderadera yothandizira mwana wathu kuwerenga (Plan and create time to support a child’s reading at home)

8.4 3.6 6.4 8.2

Note: Percentages add to more than 100 because of the option for respondents to supply more than one answer.

Malawi SBCC Endline Report 8

Grade Reading Activity. The group developed the campaign name, logo, messages, and content (such as theater sketch concepts, visuals, and radio program formats). The campaign featured a package of interventions, in line with SBCC best practices suggesting that multiple channels would be more effective than a campaign using a single channel.

Radio: Thirteen radio programs were broadcast between June 8 and September 3, 2016, on Radio Bembeke, a local station with good coverage in Kasinje, the intervention zone, but no coverage in Senzani, the comparison zone. Each 30-minute program was broadcast on Wednesdays with a repeat on Saturdays. In addition, three 1-minute public service announcements spots were produced and aired periodically on Radio Bembeke during July through September, for a total of 372 airings.

Radio listening clubs: Based on preliminary data suggesting that radio listenership was relatively low (at baseline, 39 percent of Kasinje respondents said they never listened to the radio), radio listening clubs were created to widen listenership. A total of 204 leaders and participants were trained in May, and each club was given a solar-powered radio.

During the recall section of the endline survey, 18 percent of respondents in Kasinje said they had attended radio listening clubs or community meetings about their child’s education. Of those, 88 percent said they learned educational activities that they could do with their children, and of that group, 90 percent said they did activities with their children at home.

Posters: Five poster designs were produced, and a total of 180 posters were posted on focal points such as markets, schools, and trading centers in the intervention zone.

Drama: The community engaged in the theater activities. For example, during 15 performances in June, attendance was 1,095 parents, 1,422 schoolchildren, 165 community leaders.

Box: Community Involvement Monitoring

Early Grade Reading Activity staff visited the intervention zone Aug. 4–8 during the school holiday to monitor implementation and gather some information on reach and impact. The team visited 75 households with children in Standards 1, 2, and 3, and asked the following questions:

• Did your Standard 1–3 child bring home reading materials from school in the past term?

• How often does your child read outside of school?

• Have you ever listened to the Early Grade Reading Activity SBCC radio program on Radio

Bembeke?

• Have you ever practiced what you have learned from the radio program or drama performance?

• Were you reached out to by the SBCC radio listener clubs members?

• This holiday, did you borrow books from the school so that you could continue assisting your child with reading?

• Who assists your child with reading outside of school?

Results

• Of the 75 households interviewed, the breakdown was as follows:

o 38 parents whose children were in Standard 1 last term

Malawi SBCC Endline Report 9 o 23 parents whose children were in Standard 2 last term o 14 parents whose children were in Standard 3 last term

• 81.3 percent stated that their children were bringing books home last term

• On average, the learners were reading books at home four days per week

• 41 percent had listened to the SBCC radio program on Bembeke community radio

• 67 percent had attended the SBCC drama performance

• 65 percent had practiced what they had heard on the radio or learned from the drama performance

• 59 percent were reached out to by the radio listening club members

• 49 percent had borrowed books from their respective schools during the holiday

3.2 Attitudes, Norms and Self-Efficacy

This study was designed to measure not only changes in behavior as a result of the SBCC campaign but also changes in attitudes, norms, and perceptions of self-efficacy—all of which are known to strongly influence behavior, as illustrated by the Integrative Model proposed by Fishbein (2000) and refined by Fishbein and Cappella (2006). This model brings together several commonly cited behavioral theories and serves to predict and explain behavior by illustrating relationships among the factors that influence whether or not a person performs a desirable behavior, such as handwashing; or stops an undesirable behavior, such as smoking (see Figure 2).

Figure 2. Integrative Model of Behavior Change

Source: Fishbein & Cappella (2006).

Malawi SBCC Endline Report 10

The goal of an SBCC campaign is to alter behavior, which is influenced by environmental factors, skills and abilities, and intention. However, communication alone primarily acts on the three factors that influence intention in the Integrative Model:

• Attitudes (a person’s overall favorable or unfavorable feelings toward the behavior);

• Norms (perceptions of what others think and perceptions of what others are doing); and

• Self-efficacy (confidence in one’s ability to perform the behavior, even under difficult circumstances).

Each of these three factors is influenced by the person’s beliefs, and beliefs are the most effective target for persuasive communication (Fishbein & Cappella, 2006). For example, a belief that influences attitudes might be, “my child will do better in school if she learns to read well by second grade,” or “my child should not spend time reading for pleasure when there are chores to be done.” Alternatively, a normative belief would be, “my neighbors will think I am a bad mother if I don’t read with my child,” while a control belief would claim, “I don’t know how to read so there is nothing I can do to help my child learn to read.” The results for the relevant survey questions are, therefore, presented in three sections: attitudes, norms, and perceptions of self-efficacy.

As described above in Section 2.2, “Analytical Approach,” results of the endline survey were analyzed using propensity score matching, and reported as odds ratios. We analyzed the results in three ways. The first compared responses before the campaign to responses given after the campaign in Kasinje, the treatment area: The odds of a post-campaign respondent giving a response were compared to the odds of a matched pre-campaign respondent giving the same response. The second analysis used the same method to compare endline results in the treatment zone with endline results in the comparison zone: The odds of a post-campaign Kasinje (treatment) respondent giving a response were compared to the odds of a matched post-campaign Senzani (comparison) respondent giving the same response. The third analysis examined the “dose-response” effect of the campaign in Kasinje: Respondents were divided into two groups based on how many exposures to the campaign they reported, and the odds of a high-exposure respondent giving a response were compared to the odds of a matched low-exposure respondent giving the same response. The odds ratios are presented in figures, with statistically significant results shown in dark blue. Tables are also presented showing the percentages of respondents who gave a particular response.

3.2.1 Attitudes

As noted above, Fishbein and Cappella’s (2006) Integrative Model defined attitudes as a function of behavioral beliefs and outcome evaluations. That is, “the more one believes that performing the behavior in question will lead to ‘good’ outcomes and prevent ‘bad’ outcomes, the more favorable should be one’s attitude toward performing the behavior” (Cappella, Yzer, & Fishbein, 2003, p. 211). Results of the formative assessment suggested that parents had positive attitudes toward education. In addition, they believed that children whose parents paid

Malawi SBCC Endline Report 11 attention to their schooling would be more successful, and that success in school would lead to success in life. The survey’s “attitude” questions therefore focused on the role parents could play to help their child with school, and parents’ beliefs about the importance of reading compared to other school subjects.

Respondents were asked what role a parent (not necessarily in their household) could play to help their children with school. The responses were unprompted; that is, the respondents were not given a list of possible answers, and they could give multiple responses. When pre-campaign and post-campaign responses were analyzed for Kasinje, three responses increased significantly: do homework with the child (OR: 1.69); read with the child (OR: 2.37); and check the child’s homework (OR: 1.89; see Figure 3). Several behaviors that were not promoted during the campaign showed a decrease. When endline results for the two zones were compared, “Read with the child” was significantly higher in Kasinje (OR: 1.41). Other responses either showed no difference or were higher in Senzani. The exposure analysis showed significant differences between high-exposure and low-exposure respondents in Kasinje for the main behaviors promoted by the campaign. A high-exposure respondent was twice as likely to cite “have the child read aloud” (OR: 2.10), “read with the child” (OR: 2.01), and “check the child’s notebook (OR: 2.10), and the odds were almost as high for “check the child’s homework” (OR: 1.67). Another response that increased significantly was “buy books and workbooks” (OR:

2.95). No responses were lower in the high-exposure group.

Figure 3. Kasinje: Attitudes – Role parents can play to help their child with school (unprompted)

Malawi SBCC Endline Report 12

Table 3. Kasinje: Attitudes – Role parents can play to help their child with school (unprompted) (percentages)

Question (unprompted): “What role can parents play to help their child with school?”

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

Meet with teacher 1 1 3 4

Meet with head teacher 0 0 1 2

Check child’s notebook 36 39 43 49

Check child’s homework * † 11 20 12 32

Make sure the child has a good breakfast * † 17 12 26 18

Read with the child * † 22 40 18 34

Have the child read aloud to you 9 8 9 13

Do homework with the child * † 8 13 11 19

Hire a tutor if the child is not doing well in school 1 1 4 3

Attend school assemblies † 0 0 1 3

Buy school supplies * 20 7 20 16

Buy books and workbooks * 10 4 9 12

Make sure the child attends school on time * † 56 25 49 31

Other (specify) 24 17 36 14

* Change from Kasinje baseline to Kasinje endline: significant p-value < 0.05.

Difference between Kasinje endline and Senzani endline: significant p-value < 0.05.

† Change from Senzani baseline to Senzani endline: significant p-value < 0.05.

3.2.2 Norms

To better understand the perceptions of local norms, the respondents were asked, “Do you know of friends and neighbors who read with their children?” In Kasinje, the proportion of those who said “yes” increased from 47 percent at baseline to 60% at endline (OR: 1.53; see Figure 4 and Table 4). The odds of a Kasinje resident saying “yes” at endline were 1.45 times the odds of a Senzani respondent saying “yes” at endline (OR: 1.45). When analyzed by exposure (see Table 5), the odds of a high-exposure Kasinje respondent saying “yes” were almost twice those of a low-exposure Kasinje resident (OR: 1.86).

Malawi SBCC Endline Report 13

Figure 4. Kasinje: Norms

Table 4. Kasinje: Norms, pre vs. post (percentages)

Question

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

Do you know of friends or neighbors who read with their children? *

47 60 44 49

* Change from Kasinje baseline to Kasinje endline: significant p-value < 0.05.

Table 5. Kasinje: Norms by exposure (percentages)

Question Lowest exposure category Highest exposure category

Do you know of friends or neighbors who read with their children?

53 67

Note: Differences between low-exposure Kasinje and high-exposure Kasinje: significant p-value < 0.05.

Malawi SBCC Endline Report 14

3.2.3 Self-Efficacy

To try to determine household members’ sense of self-efficacy, the survey asked them if they felt they could help their child learn to read. Responses were high at baseline: 70 percent in Kasinje and 77 percent in Senzani said “yes” (Table 6). In Kasinje, the proportion of those who said “yes” increased from 70 percent at baseline to 77% at endline (OR: 1.40; see Figure 5).

There was no difference between responses in the two zones at endline. When analyzed by exposure, 88 percent of high-exposure residents said yes, compared to 66 percent of low-exposure residents (Table 7). The odds of a high-exposure Kasinje respondent saying “yes” were almost four times that of a low-exposure Kasinje resident (OR: 3.90).

For those who answered yes, a follow-up question asked them to name ways they thought they could do so (Table 6). Responses in Kasinje showed no change from baseline to endline. The odds of a Kasinje resident saying “reading with the child” at endline were somewhat higher than the odds of a Senzani respondent giving the same answer (OR: 1.37). The odds of a high-exposure Kasinje respondent were more than twice the odds for a low-exposure Kasinje respondent for “making time for child to study” (OR: 2.50) and “having child read aloud” (OR:

2.49). The odds of a high-exposure resident saying “reading with child” were almost three times those of a low-exposure resident (OR: 2.85).

Figure 5. Kasinje: Self-efficacy Among those who said they felt they could help their child with reading, the proportion who cited specific ways they could help.

Malawi SBCC Endline Report 15

Table 6. Kasinje: Self-efficacy, pre vs. post (percentages)

Questions

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

Do you feel you can help your child learn to read? * 70 77 77 82

If so, how?

Reading with him/her † 51 51 53 43

Having him/her read aloud to me *† 23 15 19 24

Making time for him/her to study † 24 30 24 41

* Change from Kasinje baseline to Kasinje endline: significant p-value < 0.05.

Table 7. Kasinje: Self-efficacy by exposure (percentages)

Questions Lowest exposure category Highest exposure category

Do you feel you can help your child learn to read? 66 88

If so, how?

Reading with him/her 38 64

Having him/her read aloud to me 9 21

Making time for him/her to study 20 39

Note: All differences between low-exposure Kasinje and high-exposure Kasinje: significant p-value < 0.05.

3.3 Behavior

Distinct from what they thought they could or should do, respondents were asked if they actually did help their child with schoolwork, and if so, how. The behavior questions focused on those behaviors that would support a child’s reading.

Figure 6 shows the results when respondents were asked if they helped their child with school work, and if so, how. Note the distinction between this and the “attitudes” question above, which asked in general what role a parent can play. When respondents were asked if they helped the child with school work, about two-thirds said yes; the proportions did not differ significantly between zones or from baseline to endline. However, at endline, the odds of a high-exposure Kasinje respondent saying “yes” were nearly four times those of a low-exposure respondent (OR: 3.95). When asked how often they helped, 45 percent of Kasinje respondents at endline said “every day” and 22 percent said once a week; the proportions did not change significantly from baseline to endline and were similar in both zones. When those who said they helped were asked how they helped, the high-exposure Kasinje residents were significantly more likely to

Malawi SBCC Endline Report 16 cite behaviors promoted by the campaign, such as “read to child” (OR: 2.56), “ask child to show me his/her work” (OR: 3.28) and “I ask child to read to me” (OR: 2.09).

Figure 6. Kasinje: Parent interaction with child about school

When respondents were asked if anyone in the household helped the child with school work, about half said yes; the proportions did not differ significantly between zones or from baseline to endline. When asked who helped, the most common responses in both zones were the father, sister, and brother of the learner. Thirty-three percent of respondents at endline said that a household member helped the child every day, up from 22 percent at baseline in Kasinje; this change was statistically significant. Another 51 percent of Kasinje respondents said at baseline that someone in the household helped the child two to three times a week, compared to 46 percent at endline.

When those who said a household member helped were asked to spontaneously cite how the household member helped, the pre/post analysis showed that a post-campaign Kasinje respondent had odds significantly higher than those of a pre-campaign respondent for replying “helps to practice language skills” (OR: 2.65), and “practices reading with child” (OR: 1.43; see Figure 7). Comparisons between Kasinje and Senzani at endline mostly showed no difference, although responses for two important behaviors were higher in Senzani at endline. When analyzed by exposure (Table 8), 39 percent of high-exposure Kasinje residents cited “practices reading with child” compared to 25 percent of low-exposure residents. The proportions for other behaviors were lower, but the differences were mostly statistically significant: The odds of a

Malawi SBCC Endline Report 17 high-exposure Kasinje respondent were nearly three times the odds of a low-exposure respondent saying “read stories to child” (OR: 2.89). The odds were about double for four other behaviors.

Figure 7. Kasinje: Behavior – How household members help the child with school, pre vs. post

When household members help a child with school, how do they help?

Table 8. Kasinje: Behavior – How household members help the child with school, by exposure (percentages)

Question: “When household members help a child with school, how do they help?”

Lowest exposure category

Highest exposure category

Helps with homework 9 17

Read stories to him/her 4 12

Has the child read to him/her 6 13

Practices reading with him/her 25 39

Helps to practice language skills 8 15

Note: All differences listed between low-exposure Kasinje and high-exposure Kasinje: significant p-value < 0.05.

Respondents who recalled any campaign element were asked to report the campaign’s influence on their behavior in general. When asked if the campaign had a lot of impact, a little

Malawi SBCC Endline Report 18 impact, or no impact on the likelihood that they would read with their child at home, 56 percent said “a lot of impact” (Table 9). The proportion was similar for “do educational activities with your child at home.” Sixty-seven percent said the campaign made them more likely to talk to their child about school, and 51 percent said it made them more likely to encourage other parents to read or do other educational activities with their children.

Table 9. Likelihood of reporting certain behaviors as a result of the campaign

Question % Yes n

Overall, did the campaign make you more likely to:

Read with your child at home?

No difference 34.6 156

A little impact 9.8 44

A lot of impact 55.7 251

Do other educational activities with your child at home?

No difference 35.7 161

A little impact 11.3 51

A lot of impact 53.0 239

Discuss campaign with other, or mention to others? 36.4 164

Talk to your child more about reading/school? 67.4 304

Encourage other parents to read/do educational activities with their children? 51.0 228

Talk to your child’s teacher about his/her education or reading progress? 31.3 141

4 Results of Teacher Survey A sample of teachers from Standards 1, 2, and 3 in all the schools of the treatment and comparison zones were interviewed at baseline and endline. In addition to collecting information about the teachers themselves, about the attributes of the classroom environment, and about the availability of reading materials, the survey asked teachers many of the same questions as were posed to household members.

4.1 Teacher Attitudes

Teachers were asked what role they believed parents could play in helping their child with school (Table 10). Their responses were unprompted, and they were allowed to give multiple answers. Three key behaviors promoted by the campaign—“read with the child,” “check child’s homework,” and “do homework with the child”—showed significant increases from baseline to endline in Kasinje. Other results were more variable, or not statistically significant (Table 10).

Malawi SBCC Endline Report 19

Table 10. Teachers’ perceptions of roles parents can play to help their child with school (percentages)

Question: “In your opinion, what role, IF ANY, can parents play to help their child with school?”

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

Read with the child * 38 60 30 37

Check child's homework * 18 47 45 37

Do homework with the child * 11 38 30 30

Make sure the child attends school on time 40 22 36 43

Make sure child has good breakfast † 7 16 32 9

Meet with teacher * 2 13 9 4

Buy school supplies 29 13 49 39

No significant differences:

Buy books and workbooks 27 20 19 30

Hire a tutor if the child isn't doing well 2 4 9 4

Have child read aloud to you 13 20 26 24

Check child's notebook 60 51 40 43

Note: Columns do not add to 100% because respondents could provide more than one answer.

* Change from Kasinje baseline to Kasinje endline: significant p-value < 0.05.

Teachers were asked how frequently parents visited the school. At baseline, 29% of the teachers in Kasinje and 34% in Senzani reported having parent visits at least once a month (Table 11). Both of those figures decreased at endline, while the proportion of those who visited once a week or once every two weeks increased from baseline to endline

Table 11. Frequency of parents’ visits to schools (percentages)

Question: “On average, how often do parents of the learners in your class visit the school?”

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

Once a week 18 22 9 17

Once every two weeks 9 16 4 11

Once a month 29 24 34 26

Once every 3 months 20 16 28 20

Never 11 9 9 15

Note: Columns do not add to 100% because respondents could provide more than one answer.

Malawi SBCC Endline Report 20

To further examine parental involvement in school, teachers were asked what parents wanted to talk about when they visited the school (Table 12). In Kasinje, the only change from baseline to endline was in the proportion of parents who wanted to discuss learning achievement; this figure doubled from 20 percent at baseline to 40 percent at endline. The proportion of parents who wished to discuss discipline decreased from baseline to endline in Senzani but did not change significantly in Kasinje.

Table 12. Discussion topics when parents visited schools (percentages)

Question: “When parents of the learners visit, what do they want to discuss?”

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

Discipline † 31 49 60 39

Concerns about learning achievement * 20 40 30 22

School activities 20 18 19 37

Concerns about reading achievement 20 27 23 24

Grades 16 31 38 43

Note: Columns do not add to 100% because respondents could provide more than one answer.

* Change from Kasinje baseline to Kasinje endline: significant p-value < 0.05.

4.2 Classroom Activities Promoting Reading and Availability of Reading Materials

Teachers were asked a series of questions to determine how often they assigned homework and the availability of materials for children to take home (Table 13).

Table 13. Frequency with which teachers assign homework (percentages)

Question: “On average, how often do you assign homework?”

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

Every day † 33 44 38 65

Three times a week 27 24 26 20

Twice a week 16 11 21 7

Once a week 24 16 15 9

Note: Columns do not add to 100% because respondents could provide more than one answer.

Malawi SBCC Endline Report 21

Teachers were also asked about school books, story books, and story cards available to their students (Table 14 and Table 15).

Table 14. Children taking school books home (percentages)

Questions

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

Question: “Do the children in your current class usually take school books home with them?” 98 87 87 74

For those who answered children take school books home:

“How often do they take them home?”

Every day 23 31 24 38

Three times a week 9 8 20 6

Twice a week 16 8 5 3

Once a week 39 33 37 32

Once every two weeks 9 10 5 9

Once a month 0 8 5 9

For those who answered children do not take school books home: “Why don’t they take school books home with them?” n = 1 n = 6 n = 6 n = 12

Not enough school books available 0 50 33 33

Teachers/school directors concerned books may be damaged or lost 100 0 83 17

Parents concerned books may be damaged or lost 0 17 33 25

Table 15. Children taking story books or story cards home (percentages)

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

Question: “Do the children in your current class usually take story books/story cards home with them?” 87 84 89 78

For those who answered children take story books/story cards home: “How often do they take them home?”

Every day 5 16 7 0

Three times a week 13 5 7 14

Twice a week 15 8 12 19

Once a week 62 58 45 67

Once every two weeks 0 11 19 0

Once a month 3 3 10 0

Malawi SBCC Endline Report 22

Kasinje (treatment area)

Senzani (control area)

Baseline Endline Baseline Endline

For those who answered children do not take story books/story cards home: “Why don’t they take story books/story cards home with them?”

n = 6 n = 7 n = 5 n = 10

Not enough books/cards available 67 86 60 30

Teachers/school directors concerned books/cards may be damaged or lost 17 0 20 0

Parents concerned books/cards may be damaged or lost 0 0 20 0

No books/cards available 0 0 20 20

Children are not interested in taking books/cards 17 0 0 0

5 Discussion The Tiwerenge (Let’s Read) SBCC campaign in Ntcheu district in Malawi was designed to encourage parents to support reading at home for their early-grade children. The household survey that RTI used to measure the campaign’s impact showed that, despite some implementation challenges, campaign recall was at acceptable levels, albeit much lower than the levels of recall seen in both areas where SBCC activities were piloted in Senegal. Changes in attitudes, norms, self-efficacy, and behavior were somewhat variable when Kasinje results before and after the campaign were compared, and when endline results were compared between Kasinje and Senzani. However, when results were analyzed by exposure, those respondents who had higher exposure to the campaign were frequently more likely to report positive changes. This suggests that the campaign changed behavior among many of those who were exposed to it sufficiently, and indicates that future interventions could show even more success if exposure could be increased.

Increasing exposure would not necessarily require more resources. For example, the radio station used in the intervention, Radio Bembeke, is a local station based in Dedza. It was selected in part because its signal can be heard in Kasinje, but not in Senzani, the comparison district. However, it turned out that the signal was not as strong as expected in all parts of Kasinje, and as a result it was not listened to as much as some national radio stations. Using a national radio station for the weekly radio program and the spots could significantly increase coverage. The radio listening clubs were formed to address this issue, as well as the concern of low radio ownership in Kasinje: Only one in four households reported having a radio. If these clubs could be organized more broadly, perhaps using existing groups or institutions, the reach could be increased.

As in the Senegal SBCC study, certain attitudes and behaviors promoted by the project showed notable changes, especially when analyzed by campaign exposure. High-exposure Kasinje respondents were more likely than low-exposure ones to spontaneously cite “have the child

Malawi SBCC Endline Report 23 read aloud,” “read with the child,” and check the child’s homework/notebook” when asked about a parent’s role in their child’s schooling (Figure 3). Only one other role showed a positive association in that analysis (“buy books and workbooks”). Social norms, measured by asking “do you know of friends and neighbors who read with their children,” changed positively in all three analyses, but most strongly when comparing high-exposure and low-exposure respondents (Figure 4).

The results for self-efficacy are notable because the levels were fairly high at baseline: 70 percent of Kasinje residents said they felt they could help…