Attendance_Messaging_-_DRAFT_PWS.pdf

U.S. Department of Education

Institute of Education Sciences

National Center for Education Evaluation and Regional Assistance

Performance Work Statement

IMPACT EVALUATION OF PARENT MESSAGING STRATEGIES

ON STUDENT ATTENDANCE

Draft Date: March 17, 2016

I. INTRODUCTION

The U.S. Department of Education’s (Department) Institute of Education Sciences (IES) will evaluate the impact of a low-cost technology-based parent messaging intervention on student attendance and achievement. More specifically, the evaluation will use a sequential, multiple assignment, randomized trial (SMART) design to measure the effectiveness of different initial messaging strategies, and to see if there are any benefits to following up with more intensive or augmented strategies for parents who do not respond to the initial ones. The evaluation will focus on low-performing elementary schools that serve a high percentage of disadvantaged students

(for example, schools eligible for or receiving School Improvement Grants (SIG)) and that face challenges with chronic absenteeism.

II. THE EVALUATION

A. Legislative Authority

This evaluation is authorized by the Consolidated and Further Continuing Appropriations Act, 2015 (PL 113-235), under which the Department may reserve up to five percent of SIG program funds for activities to build the capacity of states and school districts to implement the SIG program effectively.

B. Background

Student attendance is a consistent and strong predicator of student success. A recent study found that students who missed more school than their peers scored lower on the National Assessment of Educational Progress, often exhibiting academic skills one or two years behind.

Absenteeism in even the early school years has been associated with significant long-term risks. One study found that four out of five students, who were chronically absent in kindergarten and first grade

(i.e., missed 10 percent or more school days per year), were unable to read at grade-level by third

Ginsburg, A., Jordan, P., & Chang, H. (2014). Absences Add Up: How School Attendance Influences Student

Success. Washington, DC: Attendance Works.

PERFORMANCE WORK STATEMENT (DRAFT) 2

grade. Students who are unable to read at grade-level by third grade are four times more likely than those who can to drop out of high school before graduating.

Despite the importance of attendance – even among the youngest students, whose parents are most likely making decisions regarding school – chronic absenteeism is a sizeable problem. One analysis of nationwide data suggests that nearly 10 percent of kindergartners and first graders are chronically absent.3 In some communities, a quarter of all students in kindergarten through third grade are chronically absent, and this is most prevalent among low-income students.

One way to potentially improve attendance in the early grades, while also taking into account the many budgetary constraints faced by school districts, is to use a low-cost technology-based messaging intervention to provide parents with information highlighting the importance of attendance and/or “nudges” informed by behavioral science to prompt them to improve their child’s attendance. Outside of education, particularly in health-related fields, low-cost technology-based messaging interventions have successfully been used to provide parents with information (and tools) to improve outcomes for their children.

In education, low-cost technology-based messaging interventions have been tested, with some promising findings. For example, research has shown that carefully crafted text messages can be effective in prompting high school graduates to enroll in postsecondary education and college freshmen to renew their

Free Application for Federal Student Aid.6 However, the literature on education interventions of this kind is relatively limited. Few studies have focused on absenteeism, and few studies have targeted parents of elementary school students. There has generally been a greater focus on improving outcomes for pre-school, high school, and postsecondary students. The rigorous studies done in this area have also been relatively small in scale.

Questions remain about how best to use low-cost technology-based messaging interventions to provide parents with the information and/or “nudges” that will positively affect student outcomes. In particular, more research is needed regarding various aspects of message content and sequencing, and to determine when basic information is sufficient versus when communication strategies must be more intensive (e.g., augmented with personalized or interactive features). This is an important question in light of the costs associated with the latter.

Given this question and the potential for low-cost technology-based messaging interventions to

Hernandez, D.J. (2011). Double Jeopardy: How Third-Grade Reading Skills and Poverty Influence High School

Graduation. New York, NY: The Annie E. Casey Foundation.

Attendance in the Early Grades: Why it Matters for Reading. Attendance Works and the Campaign for Grade-

Level Reading, February 2014.

Balfanz, R., & Byrnes, V. (2012). Chronic Absenteeism: Summarizing What We Know From Nationally Available

Data. Baltimore, MD: Johns Hopkins University Center for Social Organization of Schools.

For examples, see: Using Health Text Messages to Improve Consumer Health Knowledge, Behaviors, and

Outcomes: An Environmental Scan. Rockville, MD: U.S. Department of Health and Human Services, Health

Resources and Services Administration, 2014.

Castleman, B., & Page, L. (2014). Summer Nudging: Can Personalized Text Messages and Peer Mentor Outreach

Increase College Going Among Low Income High School Graduates? EDPolicyWorks Working Paper, retrieved

June 18, 2015 from: http://curry.virginia.edu/uploads/resourceLibrary/9_Castleman_SummerTextMessages.pdf;

Castleman, B., & Page, L. (2014). Freshman Year Financial Aid Nudges: An Experiment to Increase FAFSA

Renewal and College Persistence. EDPolicyWorks Working Paper, retrieved June 18, 2015 from:

http://curry.virginia.edu/uploads/resourceLibrary/29_Freshman_Year_Financial_Aid_Nudges.pdf.

For example: Kraft, M., & Rogers, T. (2015). The Underutilized Potential of Teacher-to-Parent Communication:

Evidence from a Field Experiment. Economics of Education Review, 47, 49-63.

http://curry.virginia.edu/uploads/resourceLibrary/9_Castleman_SummerTextMessages.pdf http://curry.virginia.edu/uploads/resourceLibrary/29_Freshman_Year_Financial_Aid_Nudges.pdf

PERFORMANCE WORK STATEMENT (DRAFT) 3

improve student attendance, the Department is conducting an evaluation that will use a SMART design to simultaneously test multiple messaging strategies and help determine in which circumstances and for whom a more intensive messaging intervention is effective.

Findings from this evaluation will provide relevant information for educators, administrators, and policymakers focused on improving attendance during critical elementary school years, especially in high-poverty schools. The evaluation findings will also add to the research literature on how low-cost technology-based messaging interventions might best be used in educational contexts.

C. Key Evaluation Questions

The general research questions that the contractor shall answer in conducting this evaluation are as follows:

• What is the average impact on student attendance and student achievement of providing parents with information and/or other “nudges” using a low-cost technology-based messaging intervention?

• Do certain strategies regarding message content and/or sequencing have a greater impact on attendance and achievement when compared to the other strategies tested?

• For parents for whom the initial messaging intervention does not have a positive impact on student attendance, is intensifying or augmenting the intervention effective?

• How is the messaging intervention implemented? What challenges were encountered during implementation? What are the costs of implementing the intervention, and which features of the intervention are most cost effective?

D. Evaluation Approach

The primary purpose of the Department’s evaluation is to assess the impact of a low-cost technology-based messaging intervention – targeted to parents – on student attendance and achievement. To provide the most rigorous causal impacts of the intervention, this evaluation will use random assignment methods, in particular a SMART design. Parents of students attending participating elementary schools will be initially randomized into either one of two treatment conditions or a control condition. After an interim examination of student attendance data, parents will subsequently be categorized as “responders” or “non-responders” based on criteria the contractor shall recommend with the Department’s approval. Parents classified as non-responders will then be randomly assigned a second time to receive one of two types of intensified and/or augmented messaging interventions.

The evaluation will focus on low-performing elementary schools that serve a high percentage of disadvantaged students (for example, schools eligible for or receiving SIG) and that face challenges with chronic absenteeism. The evaluation sample will specifically target the parents

PERFORMANCE WORK STATEMENT (DRAFT) 4

of students from these schools who have high baseline absenteeism, using criteria that the contractor shall recommend with the Department’s approval.

Data collection will consist of student attendance records (at baseline, as well as quarterly during the intervention year), extant student achievement data (at baseline, as well as at the end of the intervention year), extant data on parent and student characteristics (at baseline), and implementation and cost information gathered from the messaging intervention system and districts. If feasible, a parent survey measuring engagement (at the end of the intervention year) will be administered.

E. Design Issues

Several concerns need to be taken into account in designing this evaluation. These include:

The specific components of the intervention and tailored research questions that the evaluation will address. There are numerous specific research questions about messaging strategies that might be examined. For example, some research suggests that when using low-cost technology-based messaging, personalization is of paramount importance. Other research suggests that simply providing basic information is enough to nudge behavior in desired directions. Given the effort and cost associated with personalization, this evaluation might shed light on this question. At the same time, addressing all possible questions of interest related to message content, sequencing, intensity, or other features of the intervention will not be possible. Decisions regarding the specific research questions and intervention components will be based on: (a) areas where there is consensus on best practices, and additional research would not be cost efficient; (b) an understanding of the messaging features experts in behavioral economics, information sciences, and other behavioral sciences (both within and outside of education) suggest are promising but where research is limited or inconclusive; and (c) consideration of the extent to which findings would provide policy relevant, actionable guidance to districts on how to engage parents in improving student attendance. The evaluation will have customized versions of the second and third general research questions (i.e., the specific research questions) enumerated above, based on the specific messaging strategies being tested.

Outcome measures. While the initial goal of the evaluation’s intervention is to improve student attendance, the ultimate goal is to improve student achievement. Types of student achievement data typically available from administrative records include state assessment scores and end-of-course grades in math and English language arts. However, comparable student achievement data may not be available for all students in the sample.

Successfully carrying out the evaluation will thus require considering the best achievement measures to collect (for example, homework and class assignment completion data might be collected along with state assessment scores and course grades), and for whom to collect them, as well as how the collected data will be analyzed.

For example, see Lavecchia, A.M., Liu, H., & Oreopoulos, P. (2014). Behavioral Economics of Education:

Progress and Possibilities. National Bureau of Economic Research Working Paper Series. Retrieved June 10, 2015 from: http://www.nber.org/papers/w20609.pdf.

http://www.nber.org/papers/w20609.pdf

PERFORMANCE WORK STATEMENT (DRAFT) 5

The Department is also interested in whether messaging parents increases engagement and investment in their children’s schooling, as this may be a mediator of impacts of student attendance and achievement. It will be necessary to consider if parent surveys are feasible, and if so what would be assessed that might meaningfully add to the evaluation.

Implementation and cost measures. It will be important to understand what type of implementation data will be most policy relevant, and how this data will be collected and analyzed. Similarly, it will be important to have a detailed plan for how cost information will be systematically and accurately collected and analyzed using research-based best practices.

The need to develop and finalize evaluation plans in a short timeframe. To be ready for implementation during the 2017-18 school year, it will be necessary for evaluation plans to be detailed and finalized in a relatively short time frame.

Quick turnaround data analysis for SMART design. A strategy for quickly collecting and analyzing attendance data, in order to categorize parents as “responders” and “non-responders” and then conduct a second random assignment for the “non-responders,” will be necessary for the successful completion of this evaluation.

Control condition and contamination. What, if any, information or “nudges” parents might receive if they are randomly assigned to the control condition is an important consideration. That is, to estimate impacts, it will be necessary to have a sufficient contrast and to minimize risks of contamination.

Power Analyses and Random Assignment. To determine the appropriate sample size for the evaluation, it will be necessary to carry out a power analysis that takes into account the multi-armed SMART design. It is important to consider what the key contrasts are and what minimum detectable effect sizes for each contrast are most reasonable for student attendance and student achievement outcomes.

F. Period of Performance

The period of performance for this requirement shall be 36 months. The first 12 months shall focus on design and recruitment, the second 12 months shall focus on implementation and data collection, and the final 12 months shall focus on data analysis and report preparation.

III. SCOPE OF WORK

Task 1—Kick-off Meeting with the Department and Progress Reports (Fixed Price)

1.1 Kick-off Meeting: The contractor’s project director and up to three key project staff members shall meet with the Contracting Officer (CO)/Contract Specialist (CS) and the Department’s

Contracting Officer’s Representative (COR) and other appropriate IES staff no later than two weeks after the effective date of the contract award. This meeting shall be at the Department’s offices in Washington, D.C. The contractor shall provide an agenda at least one week before the

PERFORMANCE WORK STATEMENT (DRAFT) 6

meeting that includes discussion of the various tasks outlined in their final proposal revision, and that allots time for the COR to provide further details concerning the focus of task specific activities. The contractor shall also come prepared to discuss the scheduling of activities, to identify any areas of concern, to suggest ways of addressing potential problems, and to review other issues related to the contract. No later than one week after the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each will be addressed as part of the contract.

1.2 Monthly Progress Reports: The contractor shall prepare monthly progress reports due no later than the 16th of each month. In these reports, the contractor shall summarize the major activities and accomplishments for the prior calendar month. In addition, the contractor shall provide information for each project task regarding significant events and/or findings, problems encountered, and staff use. The contractor shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, and identify and discuss significant deviations from the management plan. If there are deviations, the contractor shall describe the plan for resolving the problems and shall identify and discuss any decisions needed from the COR.

1.3 Monthly Manpower Reports: The contractor shall prepare monthly manpower reports due no later than the 16th of each month. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the prior calendar month, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Task 2—Refine Evaluation Design (Cost Reimbursement)

The contractor shall revise the evaluation design submitted in the final proposal revision to elaborate on, refine, and ultimately finalize plans for the various arms of the intervention and for implementing the evaluation’s SMART design.

2.1 Draft Refined Evaluation Design Memo: The contractor shall prepare a memo that lays out the evaluation design. The memo shall include each of the following components:

(a) Description of the intervention and specific evaluation research questions

The memo shall include a table or flow chart detailing the intervention (e.g., information and/or

“nudges”) that parents will receive in each of the two initial treatment conditions and in the initial control condition. That table or flow chart shall also detail the intervention that “non-responder” parents will receive in each of the two follow-up treatment conditions. This detail shall include recommendations for a set of criteria to use to determine whether a parent assigned to one of the two initial treatment conditions is a “non-responder.” This detail shall also include a proposed sequence and content for messages. All of this detail about the components of the intervention and criteria for identifying “non-responders” shall be developed and justified based on rigorous research both within and outside of education. For any personalized messages, the

PERFORMANCE WORK STATEMENT (DRAFT) 7

detail shall include how these messages will be personalized and who will be responsible for personalizing them (i.e., the contractor, message system provider, or school staff). Following the table, the memo shall describe the customized evaluation research questions to be answered by comparing the outcomes for the various messaging strategies that are part of the intervention.

The memo shall also describe research questions of interest that the evaluation will not be able to address.

(b) Messaging service provider recommendations and implementation

The memo shall identify service providers that are able to disseminate attendance-related information to parents. The memo shall describe the various features, strengths, and limitations each service provider offers and make recommendations as to which service provider should be used in the evaluation. The identified providers must all have platforms flexible enough to allow for multiple approaches to messaging, including the ability to send customized messages and for two-way communication. The contractor shall obtain agreement from the provider for services once the final refined evaluation design memo has been approved. In the event that recruited districts have already established relationships with qualified providers, the contractor shall work with that provider to implement the intervention. Following the discussion of service providers, the memo shall describe plans for ensuring that the intervention is implemented with fidelity, including a discussion of any anticipated challenges and solutions to mitigate these challenges.

(c) Randomization plan

The memo shall describe how parents will be randomly assigned in the two stages of the

SMART design. The memo shall describe how in the first stage of the SMART design parents will be randomly assigned to either one of two treatment conditions or to a control condition.

The memo shall then describe the timing of how parents initially assigned to these two treatment groups will be categorized as “responders” or “non-responders” following the first analysis of interim student attendance data. The memo shall then describe how in the second stage of the

SMART design the group of “non-responder” parents will be randomly assigned to one of two possible intensified/augmented treatments. The memo shall address any possible sources of contamination after random assignment and how any risks will be minimized.

(d) Description of data collection plans for implementation, cost analysis, and outcomes

The memo shall detail how student attendance data will be collected at baseline and quarterly during the intervention year to ensure that impacts can be estimated in a timely manner, and that interim attendance data can be analyzed expediently enough to successfully implement the evaluation’s SMART design. It shall also detail how and for which students in the evaluation sample student achievement data (e.g., assignment completion data, state assessment scores, end-of-course grades in math and English language arts) will be collected at baseline and at the end of the intervention year.

The memo shall detail plans for collecting data on parent and student background characteristics, for the purposes of examining baseline equivalence and subgroup analyses.

PERFORMANCE WORK STATEMENT (DRAFT) 8

The memo shall detail plans for collecting implementation and cost data. These plans shall attend to the type of data that will be collected, how the data will be collected, and the timing of the data collection. The plans shall be consistent with research-based best practices in the field of implementation fidelity and cost effectiveness.

All data collection instruments related to implementation fidelity, intervention costs, and outcomes shall be included with the memo.

(e) Power analysis and plans for recruitment

The memo shall include a revised power analysis based on the final proposal revision. The power analysis shall outline the key contrasts for the evaluation and what sample sizes are necessary to obtain appropriate, policy-relevant minimum detectable effect sizes. The memo shall also detail plans for ensuring that the recruited evaluation sample targets parents of students attending low-performing elementary schools with a high percentage of disadvantaged students (for example, schools eligible for or receiving SIG) that face challenges with chronic absenteeism. The plan shall include proposed criteria for how to specifically target the parents of students from these schools who have high baseline absenteeism, and what the approach will be to addressing potential issues of obtaining consent for participation. Any materials that will be used to recruit districts, schools, or parents shall also be included.

(f) Analysis approach

The memo shall include a basic plan for what statistical analyses will be conducted using the data to answer the evaluation’s research questions, including appropriate estimating equations.

The plan shall cover questions related to impacts on outcomes, as well as questions related to implementation and costs.

The draft refined evaluation design memo shall be submitted to the COR no later than 10 weeks after the contract has been awarded. The COR will provide feedback within two weeks, and the contractor shall make any revisions necessary in order to prepare for the first Technical Working

Group (TWG) meeting (Task 3) and the draft Office of Management and Budget (OMB) package (Task 4).

2.2 Materials for Sharing Design Plans with TWG: The draft refined evaluation design memo

(Task 2.1) shall serve as the basis for the PowerPoint slide presentation that the contractor shall prepare in order to solicit guidance from experts in the field (see Task 3 below). At least three weeks before the TWG meeting, the contractor shall submit to the COR the following: (1) A set of PowerPoint slides; (2) Any handouts necessary to illustrate the evaluation design or particular plans; (3) A 3-page evaluation summary abstracted from the draft refined evaluation design memo and/or the draft forms clearance package (see Task 4); (4) A list of proposed discussion questions that experts will be asked to consider; and (5) A meeting agenda.

2.3 Final Refined Evaluation Design Memo: The final refined evaluation design memo shall be a revision of the draft refined evaluation design memo (Task 2.1) that fully incorporates feedback from the TWG discussed in Task 3 and from the COR on Tasks 2.1, 2.2, and 3. All changes shall

PERFORMANCE WORK STATEMENT (DRAFT) 9

be reflected in the final memo. For example, changes to the specific research questions the multi-armed and sequential approaches (SMART design) will be used to answer, the sequencing or content of messages, the outcomes to be measured, etc. shall all be described.

The final refined evaluation design memo shall be submitted to the COR no later than 4 weeks after the first TWG meeting. The COR will provide feedback within two weeks, and the contractor shall make any revisions necessary in order to prepare the final OMB package (Task

4).

Task 3—Technical Working Group (TWG) Meeting #1 (Design) (Fixed Price)

The contractor shall convene a group of experts who will provide advice and guidance concerning the evaluation’s methodology, the specific research questions that the evaluation will address, the information strategies that will be tested, implementation of the intervention, and interpretation of findings.

3.1a Bios for Recommended Participants: The contractor shall recruit a team of eight TWG members whose expertise areas include (but are not limited to): student attendance and absenteeism research and policy, behavioral economics/psychology and information sciences, SMART design evaluation methods, and cost effectiveness analysis methods. The contractor shall ensure that the TWG includes at least two members with expertise in designing experiments that use low-cost technology-based messaging interventions to impact education or health outcomes. No later than four weeks after the contract has been awarded, the contractor shall submit a list of potential TWG members that includes a brief description of each proposed expert’s credentials and specific area of expertise, and specifically details how the expert’s credentials and expertise align with the evaluation’s needs. The COR will review and approve the list and/or provide comments within one week.

3.1b Letters of Commitment: When the proposed TWG members are approved by the COR, the contractor shall contact each proposed member to request a signed letter of commitment for participation in the first TWG meeting (see Task 3.2) and the second TWG meeting (see Task

9.1).Signed letters of commitment shall be submitted to the COR no later than 10 weeks after the contract has been awarded.

3.2 Convene First TWG Meeting: The contractor shall convene one full day meeting of the TWG no later than 18 weeks after the contract has been awarded. The meeting shall cover the content of the draft refined evaluation design memo (Task 2.1) and specifically focus on discussing the evaluation design and getting feedback on the specific plans for messaging parents (e.g., the contractor shall be prepared to share the proposed content and sequencing for messages that will be sent as part of each treatment arm). The meeting shall take place in The Department facilities in Washington, DC, and the contractor shall cover the travel, per diem, and honorarium expenses of the TWG members. Food and beverage shall not be provided for attendees, but the contractor shall build in a coffee break and a lunch break into the agenda. The 3-page evaluation summary, list of proposed discussion questions that experts will be asked to consider (Task 2.2), and meeting agenda and logistical information shall be distributed to panel members at least one week before the meeting.

PERFORMANCE WORK STATEMENT (DRAFT) 10

3.3 Summary Memo: No later than one week after the panel meeting, the contractor shall submit a written summary memo to the COR, detailing the panel’s recommendations, remaining issues, and the contractor’s proposed actions.

Task 4—Forms Clearance Package (Cost Reimbursement)

The contractor shall prepare all forms and documents necessary for OMB clearance and shall coordinate its own Institutional Review Board (IRB) approval.

4.1 Draft OMB Package: The contractor shall prepare a draft OMB forms clearance package that includes all required forms, calculations of participant burden, and data collection materials and instruments. The package shall incorporate: (1) a description of the evaluation’s general purpose,

(2) the general and specific research questions, (3) a description of the intervention and evaluation design, including a crosswalk that illustrates how the various components of the intervention will allow the evaluation to address the specific research questions, (4) recruitment materials, (5) a description of and justification for any incentives, (6) limitations of the evaluation and its potential use for policy decisions, and (7) a description indicating how data will be tabulated and analyzed.

The first draft of the OMB package shall be submitted to the COR no later than 20 weeks after contract award. The COR will provide feedback within one week, and the contractor shall make any revisions necessary in order to prepare the OMB package for initial submission 22 weeks after contract award.

4.2 Revised OMB Package: Based on comments received after internal and external review of the initially submitted OMB package -- gathered during the first 60 calendar days of the clearance process -- the contractor shall make all needed revisions and submit a revised OMB package to the COR. The contractor shall be prepared to make one additional round of revisions to the OMB package as required during the latter stage of the clearance process. The contractor shall keep in mind that clearance requires a minimum of 120 calendar days.

4.3 Documentation of IRB Approval: The contractor shall coordinate its own IRB approval for the data collection, ensuring that all aspects of the evaluation have been approved.

Documentation of the contractor’s IRB approval shall be submitted to the COR no later than 30 weeks after the contract has been awarded.

Task 5—Recruit Evaluation Sample (Cost Reimbursement)

The contractor shall implement the recruitment plan developed in Task 2, and ensure that a sufficient sample size is identified and recruited to participate in the evaluation. The contractor shall ensure that the participating elementary schools are low-performing and serve a high percentage of disadvantaged students (for example, schools eligible for or receiving SIG). The contractor shall also ensure that the sample targets parents of students with high levels of baseline absenteeism, as specified in the final refined evaluation design memo (Task 2.3). The

PERFORMANCE WORK STATEMENT (DRAFT) 11

contractor shall ensure that parental consent to be randomly assigned to receive the messaging intervention is obtained prior to conducting the random assignment.

5.1 Recruitment Materials: For recruitment purposes, the contractor shall prepare a non-technical brochure describing the evaluation that is suitable for distribution to a broad audience of policymakers and educators. The contractor shall also include recruitment letters geared towards districts and schools that include general information on the evaluation, as well as specific information on the importance of the evaluation, its purposes, products, timelines, and the expected benefits to be derived from participation. If necessary, the contractor shall devise a way for parents to provide informed consent while avoiding experimental contamination.

The contractor shall submit a draft brochure, draft letters, and all other notification materials to the COR for review. These materials shall be submitted along with the draft OMB package (see

Task 4.1) no later than 20 weeks after contract award. The COR will provide feedback within one week, and the contractor shall make any revisions necessary in order to have the materials ready for the initial OMB package submission.. Once notified by the COR, the contractor shall send the materials to districts/schools and follow-up with the necessary telephone calls and/or site visits to secure participation in the evaluation.

5.2 Memoranda of Understanding (MOU): The contractor shall establish and sign a formal MOU with each participating school district. The MOU shall indicate the roles, responsibilities, and expectations of each party. The MOU shall include a specific agreement regarding when districts will make available interim attendance data so that the second randomization can be implemented. The MOU shall be signed by the contractor and the lead school district staff. The

MOU shall include, but not be limited to, the following topics: the intervention; the proposed evaluation design and work plan; random assignment procedures; implementation of the intervention; data needs; compensation to offset financial burdens associated with participation in the evaluation. The contractor shall have MOUs signed no later than August 1, 2017.

As part of the recruitment process, the contractor shall comply with: The Privacy Act of 1974, P.L. 93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and Privacy Act of

1974, 20 USC 1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41 CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the

Federal common rule or the Department’s final regulations on the protection of human research participants.

5.3 Recruitment Progress Reports: During recruitment (roughly May or June 2017-August 2017), the contractor shall submit a brief update to the COR that details progress, problems, and plans to address any problems on a biweekly basis, specifically on the 16 th and 30 th of each month or the

Monday following these dates if they fall over a weekend.

Task 6—Randomize and Implement Intervention (Cost Reimbursement)

The contractor shall randomly assign parents who are participating in the evaluation into either one of the two initial treatment arms or into the control condition. The contractor shall then implement the initial intervention as specified in the final refined evaluation design memo (Task

2.3). The contractor shall work with districts and schools to obtain timely interim attendance data

PERFORMANCE WORK STATEMENT (DRAFT) 12

in order to categorize parents as “responders” and “non-responders.” Once “non-responders” have been identified, the contractor shall randomly assign these parents to receive one of two possible intensified/augmented messaging interventions, as specified in the final refined evaluation design memo (Task 2.3). The contractor shall then implement the intensified/augmented interventions, as well as continue to implement the initial intervention to the “responder” parents. The contractor shall ensure faithful implementation of the messaging intervention during the course of the intervention school year. The contractor shall monitor implementation for fidelity and, as needed facilitate implementation by working with the messaging service provider and districts.

The messages shall be sent out during the 2017-18 school year. It is expected that the second random assignment of “non-responder” parents shall occur in December 2017 so that the second phase of message implementation can coincide with the end of most students’ winter break.

6.1 Implementation Progress Reports: During implementation (2017-18 school year), the contractor shall submit a brief update to the COR that details progress, problems, and plans to address any problems on a biweekly basis, specifically on the 16 th and 30 th of each month or the

Monday following these dates if they fall over a weekend.

Task 7—Data Collection (Cost Reimbursement)

The contractor shall follow the plan described in the final refined evaluation design memo (Task

2.3) and gather all documents and information necessary to carry out a high-quality formal cost-effectiveness analysis, consistent with research-based best practices, in order to answer the evaluation’s fourth research question.

2.3) and gather all information necessary to assess implementation fidelity of the intervention.

For example, from the messaging system, the contractor may obtain implementation information such as the number of messages sent to parents, the number of messages received by parents, and any other information available.

2.3) and gather from the participating districts and/or schools student attendance data on a rolling basis, so as to timely determine: (1) whether students have a high baseline level of absenteeism,

(2) whether parents of the students are “non-responders” to the initial treatment intervention, and

(3) whether the intervention had impacts on student attendance at the end of the intervention year. The contractor shall also follow the plan and gather the necessary student- and parent-level baseline characteristics that will be necessary for analysis, as well as all relevant extant student achievement data (e.g., assignment completion data, state assessment scores, end-of-course grades).

The contractor, and all subcontractors, shall comply with: The Privacy Act of 1974, P.L. 93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and Privacy Act of 1974, 20 USC

1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not

PERFORMANCE WORK STATEMENT (DRAFT) 13

limited to: 41 CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the Federal common rule or the Department’s final regulations on the protection of human research participants.

The contractor shall be required to establish procedures at its sites and at subcontractor sites to ensure confidentiality of data. Their systems shall be required to provide reasonable assurance that information identifying individuals is in files physically separated from other research data.

The contractor and subcontractors shall be required to maintain security of the complete set of all master data files and documentation. Access to individually identifiable data shall be strictly controlled. At each site all data shall be kept in locked file cabinets during nonworking hours, and work on hardcopy data shall take place in a single room, except for data entry. Physical security of electronic data shall also be maintained. Security features that protect project data shall include password-protected accounts that authorize users to use the contractor’s and subcontractors’ information systems but only to access specific network directories and network software; user rights and directory and file attributes that limit those who can use particular directories and files and determine how they can use them; and, additional security features that the network administrators establish for projects as needed. Contractor and subcontractor employees who “maintain” (collect, maintain, use, or disseminate) such data shall comply with the requirements of the confidentiality standards in section 183 of the ESRA (20 U.S.C. 9573).

In safeguarding personally identifiable information (PII), the contractor and its subcontractors also shall be subject to the Department’s requirements contained in the Department’s Handbook for the Protection of Sensitive But Unclassified Information, OCIO-15, and the Department’s policy that the transmission of sensitive but unclassified information, including PII, through an e-mail requires that the contents be password protected in a ZIP file.

All persons who collect, code, or otherwise have access to raw data must be trained in confidentiality issues and shall provide signed assurances of nondisclosure of confidential information. The contractor shall maintain these records and provide them to the Government upon request. When possible, the contractor shall ensure that individuals collecting and/or coding data do not have knowledge of the participant’s treatment condition.

In accord with OMB Memorandum M-05-04, and with the NIST SP 800-44, all publicly accessible Federal websites and web services must have HTTPS enabled, and shall only provide service over a secure connection, and e-mail applications must have SMTP enabled.

In order to comply with the Federal Records Act of 1950, as amended (44 U.S.C. 3101-3107), the Contractor shall certify to the COR that each staff member has completed the Department's basic records management training within 10 business days of starting work on the contract.

Compliance with The Department’s IT Security Policy

The contractor, and all sub-contractors, shall comply with the Department of Education’s IT security policy requirements, specifically those set forth in the ‘Handbook for Information

Assurance Security Policy (OCIO-01)’, and other applicable procedures and guidance. The contractor, and all sub-contractors, shall develop and implement management, operational and technical security controls to assure required levels of protection for information systems. The

PERFORMANCE WORK STATEMENT (DRAFT) 14

contractor, and all sub-contractors, shall further comply with all applicable Federal IT security requirements including, but not limited to, the Federal Information Security Management Act

(FISMA) of 2002, Office of Management and Budget (OMB) Circular A-130 Appendix III, Homeland Security Presidential Directives (HSPD), the National Institute of Standards and

Technology (NIST) standards and guidance, and the Federal Risk and Authorization

Management Program (FedRAMP) requirements and guidance.

These security requirements include, but are not limited to, the successful Security Authorization

(SA) of the system (includes commercially owned and operated systems managed by the commercial vendor and its sub-contractors, supporting Department programs, contracts, and projects); obtaining a full Authority to Operate (ATO) before being granted operational status;

performance of annual self-assessments of security controls; annual Contingency Plan testing;

performance of periodic vulnerability scans; updating all information system security documentation as changes occur; and other continuous monitoring activities, which may include, mapping, penetration and other intrusive scanning. Full and unfettered access for the

Department’s third party Managed Security Services Provider (MSSP) must be granted to access all computers and networks used for this system. Additionally, when there is a significant change to the system’s security posture, the system (Federal and commercial prime- and sub- contractors included) must have a new SA, with all required activities to obtain a new ATO, signed by the

Authorizing Official (AO).

System security controls shall be designed and implemented consistent with NIST SP 800-53

Rev 3, ‘Recommended Security Controls for Federal Information Systems and Organizations.’

All NIST SP 800-53 controls must be tested / assessed no less than every 3 years, according to federal and Department policy. The risk impact level of the system will be determined via the completion of the Department's inventory form and shall meet the accurate depiction of security categorization as outlined in Federal Information Publishing Standards (FIPS) 199, ‘Standards for Security Categorization of Federal Information and Information Systems.’

System security documentation shall be developed to record and support the implementation of the security controls for the system. This documentation shall be maintained for the life of the system. The contractor, and all sub-contractors, shall review and update the system security documentation at least annually and after significant changes to the system, to ensure the relevance and accurate depiction of the implemented system controls and to reflect changes to the system and its environment of operation. Security documentation must be developed in accordance with the NIST 800 series and Department of Education policy and guidance.

The contractor, and all sub-contractors, shall allow Department employees (or Department designated third party contractors) access to the hosting facility to conduct SA activities to include control reviews in accordance with NIST SP 800-53, Rev. 3 and NIST SP 800-53A. The contractor, and all sub-contractors, shall be available for interviews and demonstrations of security control compliance to support the SA process and continuous monitoring of system security. In addition, if the system is rated as ‘Moderate’ or ‘High’ for FIPS 199 risk impact, vulnerability scanning and penetration testing shall be performed on the hosting facility and application as part of the SA process. Appropriate access agreements will be reviewed and signed before any scanning or testing occurs.

PERFORMANCE WORK STATEMENT (DRAFT) 15

Identified deficiencies between required NIST SP 800-53 Rev. 4 controls and the contractor’s, and all sub-contractor’s implementation, as documented in the Risk Assessment Report, System

Security Plan (SSP) and Security Assessment Report (SAR), shall be tracked for mitigation through the development of a Plan of Action and Milestones (POA&M) in accordance with the

‘Handbook for Information Assurance Security Policy (OCIO-01).’ Depending on the severity of the deficiencies, the Department may require remediation before an ATO is issued.

All awarded contracts shall ensure that:

1. Their IT product/system is monitored during all hours of operations using entrusted detective/preventive systems;

2. Their IT product/system has current antiviral products installed and operational;

3. Their IT product/system is scanned on a reoccurring basis;

4. Vulnerabilities are remediated in a timely manner on their IT product/system; and

5. Access/view for cyber security situational awareness on their IT product/system is made available to the Department CIRC (cyber incident response capability).

Internet Protocol version 6 (IPv6)

For IPv6, the contractor shall provide COTS solutions that are IPv6 capable. An IPv6 capable system or product shall be capable of receiving, processing, transmitting and forwarding IPv6 packets and/or interfacing with other systems and protocols in a manner similar to that of IPv4.

Specific criteria to be deemed IPv6 capable are:

An IPv6 capable system that meets the IPv6 base requirements defined by the USGv6

Profile (http://www.antd.nist.gov/usgv6/profile.html).

Systems being developed, procured or acquired shall maintain interoperability with IPv4 systems/capabilities.

Systems shall implement IPv4/IPv6 dual-stack and shall also be built to determine which protocol layer to use depending on the destination host it is attempting to communicate with or establish a socket with. If either protocol is possible, systems shall employ IPv6.

The contractor shall provide IPv6 technical support for system development, implementation and management. System Development Standards: Information systems shall be developed in accordance with the Department’s Lifecycle Management Framework (LCM), ACS-OCIO 1-10.

Reporting of Data Security Breaches

If there is a suspected or known breach/disclosure of PII due to lost, theft, intercepted transfer, or other, the contractor must ensure that this breach is reported to the agency as soon as the contractor has knowledge of it. Per Office of Management and Budget Memorandum M-06-19, Federal agencies have a requirement to report breaches of PII security to a Federal incident response center. IES must notify the Department within 30 minutes of discovering the incident

(and the agency should not distinguish between suspected or confirmed breaches). The data security plan must be written to reflect this requirement, and the contractor must provide http://www.antd.nist.gov/usgv6/profile.html

PERFORMANCE WORK STATEMENT (DRAFT) 16

sufficient notification and documentation of the suspected loss, as it is understood at the time of notification to the agency for this requirement to be met. Follow-up reports of the final status of loss events will also be prepared by the contractor within a reasonable period of time as advised by the COR.

7.1 Data Collection Progress Reports: During data collection (2017-18 school year), the contractor shall submit a brief update to the COR that details progress, problems, and plans to address any problems on a biweekly basis, specifically on the 16 th and 30 th of each month or the

Monday following these dates if they fall over a weekend. The data collection progress update may accompany the implementation progress update in Task 6.1.

Task 8—Analyze Data and Produce Findings Tables (Cost Reimbursement)

The contractor shall prepare a detailed data analysis plan that clearly addresses all of the key evaluation research questions. The contractor shall then implement the data analysis plan using data collected in Task 7, and produce a set of findings tables that answer all of the key evaluation research questions.