Amendment_0001.pdf

NSN 7540-01-152-8070 STANDARD FORM 30. (Rev. 10-83) Previous Edition unusable Prescribed by GSA FAR (48 CFR) 53.243

Page of Pages AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT 1. Contract ID Code

2. AMENDMENT MODIFICATION NO. 3. EFFECTIVE DATE 4. REQUISITION/PURCHASE REQ. NO. 5. PROJECT NO. (if applicable)

6. ISSUED BY CODE 7. ADMINISTERED BY (If other than item 6) CODE

(x) 9A. AMENDMENT OF SOLICITATION NO.

9B. DATED (SEE ITEM 11)

10A. MODIFICATION OF CONTRACT/ORDER NO.

8. NAME AND ADDRESS OF CONTRACTOR (NO., Street, Country, State and ZIP Code)

CODE FACILITY CODE

10B. DATED (SEE ITEM 13)

11. THIS ITEM ONLY APPLIES TO AMENDMENTS OF SOLICITATIONS

The above numbered solicitation is amended as set forth in item 14. The hour and date specified for receipt of offers is extended, is not extended.

Offers must acknowledge receipt of this amendment prior to the hour and date specified in the solicitation or as amended, by one of the following methods:

(a) By completing items 8 and 15, and returning ______ copies of amendment; (b) By acknowledging receipt of this amendment on each copy of the offer submitted;

or (c) By separate letter or telegram which includes a reference to the solicitation and amendment numbers. FAILURE OF YOUR ACKNOWLEDGMENT TO BE RECEIVED AT THE PLACE DESIGNATED FOR THE RECEIPT OR OFFERS PRIOR TO THE HOUR AND DATE SPECIFIED MAY RESULT IN REJECTION OF YOUR OFFER. If by virtue of this amendment your desire to change an offer already submitted, such change may be made by telegram or letter, provided each telegram or letter makes reference to the solicitation and this amendment, and is received prior to the opening hour and date specified.

12. ACCOUNTING AND APPROPRIATION DATA (If required)

13. THIS ITEM ONLY APPLIES TO MODIFICATION OF CONTRACTS/ORDERS. IT MODIFIES THE CONTRACT/ORDER NO. AS DESCRIBED IN ITEM 14.

Check One

A. THIS CHANGE ORDER IS ISSUED PURSUANT TO: (Specify authority) THE CHANGES SET FORTH IN ITEM 14 ARE MADE IN THE

CONTRACT ORDER NO. IN ITEM 10A.

B. THE ABOVE NUMBERED CONTRACT/ORDER IS MODIFIED TO REFLECT THE ADMINISTRATIVE CHANGES (such as changes in paying office, appropriation date, etc) SET FORTH IN ITEM 14, PURSUANT TO THE AUTHORITY OF FAR 43.103(b).

C. THIS SUPPLEMENTAL AGREEMENT IS ENTERED INTO PURSUANT TO AUTHORITY OF:

D. OTHER (Specify type of modification and authority)

E. IMPORTANT: Contractor is not, is required to sign this document and return _______ copies to the issuing office.

14. DESCRIPTION OF AMENDMENT/MODIFICATION (Organized by UCF section headings, including solicitation/contract subject matter where feasible)

Except as provided herein, all terms and conditions of the document referenced in item 9A or 10A, as heretofore changed, remains unchanged and in full force and effect.

15A. NAME AND TITLE OF SIGNER (Type or print)

16A. NAME AND TITLE OF CONTRACTING OFFICER (Type or print)

15B. CONTRACTOR/OFFEROR 16B. UNITED STATES OF AMERICA

(Signature of person authorized to sign)

15C. DATE SIGNED

(Signature of Contracting Officer)

16C. DATE SIGNED

Attachment Page

PAGE 2 OF 29 ED-IES-16-R-0010/0001

The purpose of Amendment 0001 is: 1) to provide the U.S. Department of Education’s responses to clarification questions; 2) incorporate a revised Performance Work Statement (version date June 16, 2016); and 3) remove non-applicable clauses from the solicitation.

1. Clarification Question Responses

Responses to clarification questions received from interest parties and prospective offerors are hereby incorporated.

2. Section J.1 List of Attachments is revised as follows:

Attachment A - Messaging Performance Work Statement (version Date June 16, 2016) is hereby incorporated

3. Solicitation Clauses

The following solicitation clauses are hereby removed from the solicitation sections noted below:

Section I

I.34 52.222-17 NONDISPLACEMENT OF QUALIFIED WORKERS (MAY 2014)

I.54 52.232-11 EXTRAS (APR 1984)

Section K

K.12 310-6 DUPLICATION OF COST (MARCH 1985)

Section L

L.10 311-7 PROVISION FOR EVALUATION FACTOR AMENDMENTS (MARCH 1986)

L. 11 311-9 COMMENTS ON SMALL BUSINESS REGULATORY ENFORCEMENT (JUNE 1998)

PAGE 3 OF 29 ED-IES-16-R-0010/0001

Responses to Clarification Questions Impact Evaluation of Parent Messaging Strategies on Student Attendance

Solicitation # ED-IES-16-R-0010

1. Is there an incumbent for this work or is it a new business opportunity? If there is an incumbent, who?

U.S. Department of Education Response: This is a new requirement. It has not been completed in the past.

2. In light of the work involved in Task 2.1b to identify potential service providers, should we budget the type of formal competition for service providers typically expected in Institute for Education Sciences (IES) contracts, or does the U.S. Department of Education (the Department or ED) envision a less formal process?

U.S. Department of Education Response: The Department assumes that a formal competition for services would be the most efficient way to identify a service provider offering the necessary capabilities at the most competitive price. However, offerors may suggest alternative approaches.

(It will also be important to document the service providers districts are already working with, if any, during the initial stages of recruitment.)

3. On p. 73 there is a reference to “abstracts of current corporate work”. Does this refer to the list of contract and grant activities with the U.S. Department of Education?

U.S. Department of Education Response: Yes, the reference to “abstracts of current corporate work” on page 73 refers to the list of contract and grant activities within the U.S. Department of Education.

4. Task 7 includes data collection, but the only subtask listed is progress reports. Where should we budget the additional effort involved in collecting the multiple sources of data?

U.S. Department of Education Response: All data collection activities should be included in the budget for Task 7. The subtasks list the formal deliverables associated with each task. For the data collection task, progress reports are the only formal deliverables.

5. Finalizing the intervention likely will involve post-award effort. Should that effort be budgeted in Task 2 or 6?

PAGE 4 OF 29 ED-IES-16-R-0010/0001

U.S. Department of Education Response: The post-award effort involved in finalizing the intervention should be budgeted for in Task 2.

6. In other recent contract competitions, the Department has been explicit about hand delivery.

Is hand delivery of the final proposal an option in this Request for Proposals (RFP)?

U.S. Department of Education Response: No, hand delivery is not an option for this RFP. All proposals shall be submitted in accordance with Section L.12 of the RFP.

7. Does IES have a recommendation or guidance to offerors regarding the amount to budget for the messaging service provider?

U.S. Department of Education Response: No, IES does not have a recommendation to offerors regarding the amount to budget for the messaging service provider. This may vary depending, for example, upon the specific intervention the offeror proposes, the sample size the offeror estimates is needed, and other aspects of the proposed design.

8. On page 76, the past performance instructions state, “To support the technical proposal, offerors shall submit information about three contracts, completed in the last three years or currently in process but beyond the first year of the contract, which are of similar size, scope, complexity or, in any way, are relevant to the effort required by this solicitation. The "Contractor Performance Information” form (Attachment C) should be used to collect this information from references.” Beyond the "Contractor Performance Information” forms that will be sent directly to ED by contract references (separate from the proposal), is ED expecting bidders to provide any additional information about past performance in the technical proposal? If so, is it acceptable to place that information in an appendix?

U.S. Department of Education Response: While offerors may rely solely on references to submit the Contractor Performance Information form, the Department strongly recommends that offerors include, in the section of their proposal addressing organizational capabilities and experience, discussion of projects that are relevant to the current requirement and that include a description of the offeror’s specific role.

9. Is it acceptable to use a smaller font for tables and figures in the technical proposal? Some figures and tables are not amenable to 12 point font or would take up a large amount of space if prepared in 12 point font (for example, organization charts, timelines, staff commitment tables, GANTT charts, and the complete list of ED projects) which would reduce the information that bidders can provide to ED about important features of their technical approach.

PAGE 5 OF 29 ED-IES-16-R-0010/0001

U.S. Department of Education Response: Offerors may use alternative font sizes for tables and figures so long as these remain readable when the proposal is printed or viewed in a print ready layout at a standard paper size (8.5 x 11 inches).

10. Page 72 of the RFP states that hard copies can be sent to the address given. Can hard copies of the proposal be delivered in person on July 11th? If so, what is the procedure for hand delivering the hard copies?

U.S. Department of Education Response: No, hard copies cannot be hand delivered to the Department on July 11th. Hard copies shall be sent to the address listed in section L.12 of the

RFP.

11. What is the estimated number of labor hours for this procurement?

U.S. Department of Education Response: Please see the Forecast of ED Contract Opportunities (http://www2.ed.gov/fund/contract/find/forecast.html) for a rough estimate of the total contract value. Given that a primary goal of this evaluation is to test out a low-cost technology-based messaging intervention, the Department estimates a total contract value towards the lower end of this range.

12. The schedule of deliverables (pp. 45-47) assumes that key outcomes memo is delivered 90 weeks after the contract starting date, which would be in May 2018. However, the intervention has not yet been fully implemented at this point (e.g., in many school district the school year continues to June and the attendance data cannot be delivered before the school year is over). 1) Can ED clarify the delivery date for this deliverable? 2) Is it reasonable to assume that other deliverable dates will be pushed back accordingly?

U.S. Department of Education Response: The schedule of deliverables [and the discussion of due dates in the scope of work section of the Performance Work Statement (PWS)] miscalculated due dates, expressed in weeks from contract award, for Task 8.2 and the following tasks listed below. The PWS has been revised to reflect the corrected due dates.

Task 8.2 Key Findings Memo. PWS stated 90 weeks from contract award, should be 100 weeks from contract award. (Note, assuming a late September 2016 award, this will be an August 2018 due date.)

Task 8.3 IES Briefing. PWS stated 100 weeks from contract award, should be 109 weeks from contract award.

PAGE 6 OF 29 ED-IES-16-R-0010/0001

Task 9.1 Convene Second TWG Meeting. PWS stated 102 weeks from contract award, should be 111 weeks from contract award.

Task 10.2a Disclosure analysis plan. PWS stated 120 weeks from contract award, should be 139 weeks from contract award.

Task 10.2b Restricted-use data file. PWS stated 140 weeks from contract award, should be 152 weeks from contract award.

Please note: Offerors are always welcome to recommend an alternative schedule, so long as it is well justified.

13. The RFP request a complete list of all contrast and grants that are current or completed after November 1, 2013. Some of the larger contracting firms may have tens of contracts to be listed and even the number of relevant contracts and grants can be high and take too prohibitive amount of space in the management section. Can you provide more guidance regarding what kind of contracts must be listed?

U.S. Department of Education Response: Offerors shall provide a comprehensive listing of information about only those contracts (or grants, if the offeror does not have contracts) that are considered to be relevant to this opportunity. Offerors shall indicate whether they were the prime contractor or subcontractor for each contract (or grant, if the offeror does not have contracts) listed.

14. RFP page 73 states “The page limit includes the narrative and corresponding tables/graphics, letters of commitment, abstracts of current corporate work, and lists of the current contractual commitments of proposed staff. But then RFP 74 states” Letter(s) by proposed subcontractors/consultants indicating willingness to serve, if the offeror is selected, shall be included with the offeror's proposal as an appendix. Please clarify, does this mean that the letters can be included as an appendix, but they still count towards the narrative 50 page limit?

U.S. Department of Education Response: Letters of commitment should be included as an appendix and will NOT count towards the 50 page limit.

15. Are the current and pending time commitment tables expected to be in the narrative in the 50 page limit, or can these tables be placed in an appendix.

U.S. Department of Education Response: Current and pending time commitments for key proposed staff members are to be integrated into the proposal’s narrative as part of the “proposed

PAGE 7 OF 29 ED-IES-16-R-0010/0001

personnel” section (which is part of the 50 page limit). These tables may not be placed in an appendix.

16. Page 75 of the solicitation (Technical Proposal Format and Contents) requires that offerors provide information on Small Business Participation (such as the work to be performed, name and socioeconomic status of each small business, and the percentage of work to be performed by each small business) but no Cost information shall be provided. The solicitation also requests a Small Business Subcontracting Plan to fulfill the requirements of FAR 19.7, which includes information regarding contract values.

Can IES please confirm that the Small Business Participation information should be submitted in the technical proposal, and the Small Business Subcontracting Plan, as well as Attachment B, Subcontracting Plan Review Form, should be submitted in the business proposal?

U.S. Department of Education Response: In the Technical Proposal, offerors shall provide information on Small Business Participation (such as the name and socioeconomic status of each proposed small business, the proposed work to be performed, and percentage of work each proposed small business will perform).

In the Business Proposal, offerors shall submit the Small Business Subcontracting Plan AND the completed Subcontracting Plan Review Form (Attachment B).

17. Would IES please confirm that bidders have discretion (i.e. not limited to including any type of partner) in building the consortium?

U.S. Department of Education Response: Offerors may team or partner with a variety of entities at their discretion. There are no limitations to teaming or partnering other than those set forth in the Federal Acquisition Regulation.

PAGE 8 OF 29 ED-IES-16-R-0010/0001

Attachment A- PWS Version Date: June 16, 2016

U.S. Department of Education Institute of Education Sciences

National Center for Education Evaluation and Regional Assistance

Performance Work Statement

IMPACT EVALUATION OF PARENT MESSAGING STRATEGIES

ON STUDENT ATTENDANCE

I. INTRODUCTION

The U.S. Department of Education’s (Department) Institute of Education Sciences (IES) will evaluate the impact of a low-cost technology-based parent messaging intervention on student attendance and achievement. More specifically, the evaluation will use a sequential, multiple assignment, randomized trial (SMART) design to measure the effectiveness of different initial messaging strategies, and to see if there are any benefits to following up with more intensive or augmented strategies for parents who do not respond to the initial ones. The evaluation will focus on low-performing elementary schools that serve a high percentage of disadvantaged students (for example, schools eligible for or receiving School Improvement Grants (SIG)) and that face challenges with chronic absenteeism.

II. THE EVALUATION

A. Legislative Authority

This evaluation is authorized by the Consolidated and Further Continuing Appropriations Act, 2015 (PL 113-235), under which the Department may reserve up to five percent of SIG program funds for activities to build the capacity of states and school districts to implement the SIG program effectively.

B. Background

Student attendance is a consistent and strong predicator of student success. A recent study found that students who missed more school than their peers scored lower on the National Assessment of Educational Progress, often exhibiting academic skills one or two years behind.1 Absenteeism in even the early school years has been associated with significant long-term risks. One study found that four out of five students, who were chronically absent in kindergarten and first grade (i.e., missed 10 percent or more school days per year), were unable to read at grade-level by third

1 Ginsburg, A., Jordan, P., & Chang, H. (2014). Absences Add Up: How School Attendance Influences Student Success. Washington, DC: Attendance Works.

PAGE 9 OF 29 ED-IES-16-R-0010/0001

grade. Students who are unable to read at grade-level by third grade are four times more likely than those who can to drop out of high school before graduating.2

Despite the importance of attendance – even among the youngest students, whose parents are most likely making decisions regarding school – chronic absenteeism is a sizeable problem. One analysis of nationwide data suggests that nearly 10 percent of kindergartners and first graders are chronically absent.3 In some communities, a quarter of all students in kindergarten through third grade are chronically absent, and this is most prevalent among low-income students.4

One way to potentially improve attendance in the early grades, while also taking into account the many budgetary constraints faced by school districts, is to use a low-cost technology-based messaging intervention to provide parents with information highlighting the importance of attendance and/or “nudges” informed by behavioral science to prompt them to improve their child’s attendance. Outside of education, particularly in health-related fields, low-cost technology-based messaging interventions have successfully been used to provide parents with information (and tools) to improve outcomes for their children.5 In education, low-cost technology-based messaging interventions have been tested, with some promising findings. For example, research has shown that carefully crafted text messages can be effective in prompting high school graduates to enroll in postsecondary education and college freshmen to renew their Free Application for Federal Student Aid.6 However, the literature on education interventions of this kind is relatively limited. Few studies have focused on absenteeism, and few studies have targeted parents of elementary school students. There has generally been a greater focus on improving outcomes for pre-school, high school, and postsecondary students. The rigorous studies done in this area have also been relatively small in scale.7

Questions remain about how best to use low-cost technology-based messaging interventions to provide parents with the information and/or “nudges” that will positively affect student outcomes. In particular, more research is needed regarding various aspects of message content and sequencing, and to determine when basic information is sufficient versus when communication strategies must be more intensive (e.g., augmented with personalized or interactive features). This is an important question in light of the costs associated with the latter.

Given this question and the potential for low-cost technology-based messaging interventions to

2 Hernandez, D.J. (2011). Double Jeopardy: How Third-Grade Reading Skills and Poverty Influence High School Graduation. New York, NY: The Annie E. Casey Foundation.

3 Attendance in the Early Grades: Why it Matters for Reading. Attendance Works and the Campaign for Grade- Level Reading, February 2014.

4 Balfanz, R., & Byrnes, V. (2012). Chronic Absenteeism: Summarizing What We Know From Nationally Available Data. Baltimore, MD: Johns Hopkins University Center for Social Organization of Schools.

5 For examples, see: Using Health Text Messages to Improve Consumer Health Knowledge, Behaviors, and Outcomes: An Environmental Scan. Rockville, MD: U.S. Department of Health and Human Services, Health Resources and Services Administration, 2014.

6 Castleman, B., & Page, L. (2014). Summer Nudging: Can Personalized Text Messages and Peer Mentor Outreach Increase College Going Among Low Income High School Graduates? EDPolicyWorks Working Paper, retrieved June 18, 2015 from: http://curry.virginia.edu/uploads/resourceLibrary/9_Castleman_SummerTextMessages.pdf;

Castleman, B., & Page, L. (2014). Freshman Year Financial Aid Nudges: An Experiment to Increase FAFSA Renewal and College Persistence. EDPolicyWorks Working Paper, retrieved June 18, 2015 from:

http://curry.virginia.edu/uploads/resourceLibrary/29_Freshman_Year_Financial_Aid_Nudges.pdf.

7 For example: Kraft, M., & Rogers, T. (2015). The Underutilized Potential of Teacher-to-Parent Communication:

Evidence from a Field Experiment. Economics of Education Review, 47, 49-63.

PERFORMANCE WORK STATEMENT 2

PAGE 10 OF 29 ED-IES-16-R-0010/0001

improve student attendance, the Department is conducting an evaluation that will use a SMART design to simultaneously test multiple messaging strategies and help determine in which circumstances and for whom a more intensive messaging intervention is effective.

Findings from this evaluation will provide relevant information for educators, administrators, and policymakers focused on improving attendance during critical elementary school years, especially in high-poverty schools. The evaluation findings will also add to the research literature on how low-cost technology-based messaging interventions might best be used in educational contexts.

C. Key Evaluation Questions

The general research questions that the contractor shall answer in conducting this evaluation are as follows:

• What is the average impact on student attendance and student achievement of providing parents with information and/or other “nudges” using a low-cost technology-based messaging intervention?

• Do certain strategies regarding message content and/or sequencing have a greater impact on attendance and achievement when compared to the other strategies tested?

• For parents for whom the initial messaging intervention does not have a positive impact on student attendance, is intensifying or augmenting the intervention effective?

• How is the messaging intervention implemented? What challenges were encountered during implementation? What are the costs of implementing the intervention, and which features of the intervention are most cost effective?

D. Evaluation Approach

The primary purpose of the Department’s evaluation is to assess the impact of a low-cost technology-based messaging intervention – targeted to parents – on student attendance and achievement. To provide the most rigorous causal impacts of the intervention, this evaluation will use random assignment methods, in particular a SMART design. Parents of students attending participating elementary schools will be initially randomized into either one of two treatment conditions or a control condition. After an interim examination of student attendance data, parents will subsequently be categorized as “responders” or “non-responders” based on criteria the contractor shall recommend with the Department’s approval. Parents classified as non-responders will then be randomly assigned a second time to receive one of two types of intensified and/or augmented messaging interventions.

The evaluation will focus on low-performing elementary schools that serve a high percentage of disadvantaged students (for example, schools eligible for or receiving SIG) and that face challenges with chronic absenteeism. The evaluation sample will specifically target the parents

PERFORMANCE WORK STATEMENT 3

PAGE 11 OF 29 ED-IES-16-R-0010/0001

of students from these schools who have high baseline absenteeism, using criteria that the contractor shall recommend with the Department’s approval.

Data collection will consist of student attendance records (at baseline, as well as quarterly during the intervention year), extant student achievement data (at baseline, as well as at the end of the intervention year), extant data on parent and student characteristics (at baseline), and implementation and cost information gathered from the messaging intervention system and districts. If feasible, a parent survey measuring engagement (at the end of the intervention year) will be administered.

E. Design Issues

Several concerns need to be taken into account in designing this evaluation. These include:

• The specific components of the intervention and tailored research questions that the evaluation will address. There are numerous specific research questions about messaging strategies that might be examined. For example, some research suggests that when using low-cost technology-based messaging, personalization is of paramount importance. Other research suggests that simply providing basic information is enough to nudge behavior in desired directions. Given the effort and cost associated with personalization, this evaluation might shed light on this question. At the same time, addressing all possible questions of interest related to message content, sequencing, intensity, or other features of the intervention will not be possible. Decisions regarding the specific research questions and intervention components will be based on: (a) areas where there is consensus on best practices, and additional research would not be cost efficient; (b) an understanding of the messaging features experts in behavioral economics,8 information sciences, and other behavioral sciences (both within and outside of education) suggest are promising but where research is limited or inconclusive; and (c) consideration of the extent to which findings would provide policy relevant, actionable guidance to districts on how to engage parents in improving student attendance. The evaluation will have customized versions of the second and third general research questions (i.e., the specific research questions) enumerated above, based on the specific messaging strategies being tested.

• Outcome measures. While the initial goal of the evaluation’s intervention is to improve student attendance, the ultimate goal is to improve student achievement. Types of student achievement data typically available from administrative records include state assessment scores and end-of-course grades in math and English language arts. However, comparable student achievement data may not be available for all students in the sample.

Successfully carrying out the evaluation will thus require considering the best achievement measures to collect (for example, homework and class assignment completion data might be collected along with state assessment scores and course grades), and for whom to collect them, as well as how the collected data will be analyzed.

8 For example, see Lavecchia, A.M., Liu, H., & Oreopoulos, P. (2014). Behavioral Economics of Education:

Progress and Possibilities. National Bureau of Economic Research Working Paper Series. Retrieved June 10, 2015 from: http://www.nber.org/papers/w20609.pdf.

PERFORMANCE WORK STATEMENT 4

PAGE 12 OF 29 ED-IES-16-R-0010/0001

The Department is also interested in whether messaging parents increases engagement and investment in their children’s schooling, as this may be a mediator of impacts of student attendance and achievement. It will be necessary to consider if parent surveys are feasible, and if so what would be assessed that might meaningfully add to the evaluation.

• Implementation and cost measures. It will be important to understand what type of implementation data will be most policy relevant, and how this data will be collected and analyzed. Similarly, it will be important to have a detailed plan for how cost information will be systematically and accurately collected and analyzed using research-based best practices.

• The need to develop and finalize evaluation plans in a short timeframe. To be ready for implementation during the 2017-18 school year, it will be necessary for evaluation plans to be detailed and finalized in a relatively short time frame.

• Quick turnaround data analysis for SMART design. A strategy for quickly collecting and analyzing attendance data, in order to categorize parents as “responders” and “non-responders” and then conduct a second random assignment for the “non-responders,” will be necessary for the successful completion of this evaluation.

• Control condition and contamination. What, if any, information or “nudges” parents might receive if they are randomly assigned to the control condition is an important consideration. That is, to estimate impacts, it will be necessary to have a sufficient contrast and to minimize risks of contamination.

• Power Analyses and Random Assignment. To determine the appropriate sample size for the evaluation, it will be necessary to carry out a power analysis that takes into account the multi-armed SMART design. It is important to consider what the key contrasts are and what minimum detectable effect sizes for each contrast are most reasonable for student attendance and student achievement outcomes.

F. Period of Performance

The period of performance for this requirement shall be 36 months. The first 12 months shall focus on design and recruitment, the second 12 months shall focus on implementation and data collection, and the final 12 months shall focus on data analysis and report preparation.

III. SCOPE OF WORK

Task 1—Kick-off Meeting with the Department and Progress Reports

1.1 Kick-off Meeting: The contractor’s project director and up to three key project staff members shall meet with the Contracting Officer (CO)/Contract Specialist (CS) and the Department’s Contracting Officer’s Representative (COR) and other appropriate IES staff no later than two weeks after the effective date of the contract award. This meeting shall be at the Department’s offices in Washington, D.C. The contractor shall provide an agenda at least one week before the

PERFORMANCE WORK STATEMENT 5

PAGE 13 OF 29 ED-IES-16-R-0010/0001

meeting that includes discussion of the various tasks outlined in their final proposal revision, and that allots time for the COR to provide further details concerning the focus of task specific activities. The contractor shall also come prepared to discuss the scheduling of activities, to identify any areas of concern, to suggest ways of addressing potential problems, and to review other issues related to the contract. No later than one week after the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each will be addressed as part of the contract.

1.2 Monthly Progress Reports: The contractor shall prepare monthly progress reports due no later than the 16th of each month. In these reports, the contractor shall summarize the major activities and accomplishments for the prior calendar month. In addition, the contractor shall provide information for each project task regarding significant events and/or findings, problems encountered, and staff use. The contractor shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, and identify and discuss significant deviations from the management plan. If there are deviations, the contractor shall describe the plan for resolving the problems and shall identify and discuss any decisions needed from the COR.

1.3 Monthly Manpower Reports: The contractor shall prepare monthly manpower reports due no later than the 16th of each month. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the prior calendar month, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Task 2—Refine Evaluation Design

The contractor shall revise the evaluation design submitted in the final proposal revision to elaborate on, refine, and ultimately finalize plans for the various arms of the intervention and for implementing the evaluation’s SMART design.

2.1 Draft Refined Evaluation Design Memo: The contractor shall prepare a memo that lays out the evaluation design. The memo shall include each of the following components:

(a) Description of the intervention and specific evaluation research questions

The memo shall include a table or flow chart detailing the intervention (e.g., information and/or “nudges”) that parents will receive in each of the two initial treatment conditions and in the initial control condition. That table or flow chart shall also detail the intervention that “non-responder” parents will receive in each of the two follow-up treatment conditions. This detail shall include recommendations for a set of criteria to use to determine whether a parent assigned to one of the two initial treatment conditions is a “non-responder.” This detail shall also include a proposed sequence and content for messages. All of this detail about the components of the intervention and criteria for identifying “non-responders” shall be developed and justified based on rigorous research both within and outside of education. For any personalized messages, the

PERFORMANCE WORK STATEMENT 6

PAGE 14 OF 29 ED-IES-16-R-0010/0001

detail shall include how these messages will be personalized and who will be responsible for personalizing them (i.e., the contractor, message system provider, or school staff). Following the table, the memo shall describe the customized evaluation research questions to be answered by comparing the outcomes for the various messaging strategies that are part of the intervention.

The memo shall also describe research questions of interest that the evaluation will not be able to address.

(b) Messaging service provider recommendations and implementation

The memo shall identify service providers that are able to disseminate attendance-related information to parents. The memo shall describe the various features, strengths, and limitations each service provider offers and make recommendations as to which service provider should be used in the evaluation. The identified providers must all have platforms flexible enough to allow for multiple approaches to messaging, including the ability to send customized messages and for two-way communication. The contractor shall obtain agreement from the provider for services once the final refined evaluation design memo has been approved. In the event that recruited districts have already established relationships with qualified providers, the contractor shall work with that provider to implement the intervention. Following the discussion of service providers, the memo shall describe plans for ensuring that the intervention is implemented with fidelity, including a discussion of any anticipated challenges and solutions to mitigate these challenges.

(c) Randomization plan

The memo shall describe how parents will be randomly assigned in the two stages of the SMART design. The memo shall describe how in the first stage of the SMART design parents will be randomly assigned to either one of two treatment conditions or to a control condition.

The memo shall then describe the timing of how parents initially assigned to these two treatment groups will be categorized as “responders” or “non-responders” following the first analysis of interim student attendance data. The memo shall then describe how in the second stage of the SMART design the group of “non-responder” parents will be randomly assigned to one of two possible intensified/augmented treatments. The memo shall address any possible sources of contamination after random assignment and how any risks will be minimized.

(d) Description of data collection plans for implementation, cost analysis, and outcomes

The memo shall detail how student attendance data will be collected at baseline and quarterly during the intervention year to ensure that impacts can be estimated in a timely manner, and that interim attendance data can be analyzed expediently enough to successfully implement the evaluation’s SMART design. It shall also detail how and for which students in the evaluation sample student achievement data (e.g., assignment completion data, state assessment scores, end-of-course grades in math and English language arts) will be collected at baseline and at the end of the intervention year.

The memo shall detail plans for collecting data on parent and student background characteristics, for the purposes of examining baseline equivalence and subgroup analyses.

PERFORMANCE WORK STATEMENT 7

PAGE 15 OF 29 ED-IES-16-R-0010/0001

The memo shall detail plans for collecting implementation and cost data. These plans shall attend to the type of data that will be collected, how the data will be collected, and the timing of the data collection. The plans shall be consistent with research-based best practices in the field of implementation fidelity and cost effectiveness.

All data collection instruments related to implementation fidelity, intervention costs, and outcomes shall be included with the memo.

(e) Power analysis and plans for recruitment

The memo shall include a revised power analysis based on the final proposal revision. The power analysis shall outline the key contrasts for the evaluation and what sample sizes are necessary to obtain appropriate, policy-relevant minimum detectable effect sizes. The memo shall also detail plans for ensuring that the recruited evaluation sample targets parents of students attending low-performing elementary schools with a high percentage of disadvantaged students (for example, schools eligible for or receiving SIG) that face challenges with chronic absenteeism. The plan shall include proposed criteria for how to specifically target the parents of students from these schools who have high baseline absenteeism, and what the approach will be to addressing potential issues of obtaining consent for participation. Any materials that will be used to recruit districts, schools, or parents shall also be included.

(f) Analysis approach

The memo shall include a basic plan for what statistical analyses will be conducted using the data to answer the evaluation’s research questions, including appropriate estimating equations.

The plan shall cover questions related to impacts on outcomes, as well as questions related to implementation and costs.

The draft refined evaluation design memo shall be submitted to the COR no later than 10 weeks after the contract has been awarded. The COR will provide feedback within two weeks, and the contractor shall make any revisions necessary in order to prepare for the first Technical Working Group (TWG) meeting (Task 3) and the draft Office of Management and Budget (OMB) package (Task 4).

2.2 Materials for Sharing Design Plans with TWG: The draft refined evaluation design memo (Task 2.1) shall serve as the basis for the PowerPoint slide presentation that the contractor shall prepare in order to solicit guidance from experts in the field (see Task 3 below). At least three weeks before the TWG meeting, the contractor shall submit to the COR the following: (1) A set of PowerPoint slides; (2) Any handouts necessary to illustrate the evaluation design or particular plans; (3) A 3-page evaluation summary abstracted from the draft refined evaluation design memo and/or the draft forms clearance package (see Task 4); (4) A list of proposed discussion questions that experts will be asked to consider; and (5) A meeting agenda.

2.3 Final Refined Evaluation Design Memo: The final refined evaluation design memo shall be a revision of the draft refined evaluation design memo (Task 2.1) that fully incorporates feedback from the TWG discussed in Task 3 and from the COR on Tasks 2.1, 2.2, and 3. All changes shall

PERFORMANCE WORK STATEMENT 8

PAGE 16 OF 29 ED-IES-16-R-0010/0001

be reflected in the final memo. For example, changes to the specific research questions the multi-armed and sequential approaches (SMART design) will be used to answer, the sequencing or content of messages, the outcomes to be measured, etc. shall all be described.

The final refined evaluation design memo shall be submitted to the COR no later than 4 weeks after the first TWG meeting. The COR will provide feedback within two weeks, and the contractor shall make any revisions necessary in order to prepare the final OMB package (Task 4).

Task 3—Technical Working Group (TWG) Meeting #1 (Design)

The contractor shall convene a group of experts who will provide advice and guidance concerning the evaluation’s methodology, the specific research questions that the evaluation will address, the information strategies that will be tested, implementation of the intervention, and interpretation of findings.

3.1a Bios for Recommended Participants: The contractor shall recruit a team of eight TWG members whose expertise areas include (but are not limited to): student attendance and absenteeism research and policy, behavioral economics/psychology and information sciences, SMART design evaluation methods, and cost effectiveness analysis methods. The contractor shall ensure that the TWG includes at least two members with expertise in designing experiments that use low-cost technology-based messaging interventions to impact education or health outcomes. No later than four weeks after the contract has been awarded, the contractor shall submit a list of potential TWG members that includes a brief description of each proposed expert’s credentials and specific area of expertise, and specifically details how the expert’s credentials and expertise align with the evaluation’s needs. The COR will review and approve the list and/or provide comments within one week.

3.1b Letters of Commitment: When the proposed TWG members are approved by the COR, the contractor shall contact each proposed member to request a signed letter of commitment for participation in the first TWG meeting (see Task 3.2) and the second TWG meeting (see Task 9.1).Signed letters of commitment shall be submitted to the COR no later than 10 weeks after the contract has been awarded.

3.2 Convene First TWG Meeting: The contractor shall convene one full day meeting of the TWG no later than 18 weeks after the contract has been awarded. The meeting shall cover the content of the draft refined evaluation design memo (Task 2.1) and specifically focus on discussing the evaluation design and getting feedback on the specific plans for messaging parents (e.g., the contractor shall be prepared to share the proposed content and sequencing for messages that will be sent as part of each treatment arm). The meeting shall take place in The Department facilities in Washington, DC, and the contractor shall cover the travel, per diem, and honorarium expenses of the TWG members. Food and beverage shall not be provided for attendees, but the contractor shall build in a coffee break and a lunch break into the agenda. The 3-page evaluation summary, list of proposed discussion questions that experts will be asked to consider (Task 2.2), and meeting agenda and logistical information shall be distributed to panel members at least one week before the meeting.

PERFORMANCE WORK STATEMENT 9

PAGE 17 OF 29 ED-IES-16-R-0010/0001

3.3 Summary Memo: No later than one week after the panel meeting, the contractor shall submit a written summary memo to the COR, detailing the panel’s recommendations, remaining issues, and the contractor’s proposed actions.

Task 4—Forms Clearance Package

The contractor shall prepare all forms and documents necessary for OMB clearance and shall coordinate its own Institutional Review Board (IRB) approval.

4.1 Draft OMB Package: The contractor shall prepare a draft OMB forms clearance package that includes all required forms, calculations of participant burden, and data collection materials and instruments. The package shall incorporate: (1) a description of the evaluation’s general purpose,

(2) the general and specific research questions, (3) a description of the intervention and evaluation design, including a crosswalk that illustrates how the various components of the intervention will allow the evaluation to address the specific research questions, (4) recruitment materials, (5) a description of and justification for any incentives, (6) limitations of the evaluation and its potential use for policy decisions, and (7) a description indicating how data will be tabulated and analyzed.

The first draft of the OMB package shall be submitted to the COR no later than 20 weeks after contract award. The COR will provide feedback within one week, and the contractor shall make any revisions necessary in order to prepare the OMB package for initial submission 22 weeks after contract award.

4.2 Revised OMB Package: Based on comments received after internal and external review of the initially submitted OMB package -- gathered during the first 60 calendar days of the clearance process -- the contractor shall make all needed revisions and submit a revised OMB package to the COR. The contractor shall be prepared to make one additional round of revisions to the OMB package as required during the latter stage of the clearance process. The contractor shall keep in mind that clearance requires a minimum of 120 calendar days.

4.3 Documentation of IRB Approval: The contractor shall coordinate its own IRB approval for the data collection, ensuring that all aspects of the evaluation have been approved.

Documentation of the contractor’s IRB approval shall be submitted to the COR no later than 30 weeks after the contract has been awarded.

Task 5—Recruit Evaluation Sample

The contractor shall implement the recruitment plan developed in Task 2, and ensure that a sufficient sample size is identified and recruited to participate in the evaluation. The contractor shall ensure that the participating elementary schools are low-performing and serve a high percentage of disadvantaged students (for example, schools eligible for or receiving SIG). The contractor shall also ensure that the sample targets parents of students with high levels of baseline absenteeism, as specified in the final refined evaluation design memo (Task 2.3). The

PERFORMANCE WORK STATEMENT 10

PAGE 18 OF 29 ED-IES-16-R-0010/0001

contractor shall ensure that parental consent to be randomly assigned to receive the messaging intervention is obtained prior to conducting the random assignment.

5.1 Recruitment Materials: For recruitment purposes, the contractor shall prepare a non-technical brochure describing the evaluation that is suitable for distribution to a broad audience of policymakers and educators. The contractor shall also include recruitment letters geared towards districts and schools that include general information on the evaluation, as well as specific information on the importance of the evaluation, its purposes, products, timelines, and the expected benefits to be derived from participation. If necessary, the contractor shall devise a way for parents to provide informed consent while avoiding experimental contamination.

The contractor shall submit a draft brochure, draft letters, and all other notification materials to the COR for review. These materials shall be submitted along with the draft OMB package (see Task 4.1) no later than 20 weeks after contract award. The COR will provide feedback within one week, and the contractor shall make any revisions necessary in order to have the materials ready for the initial OMB package submission.. Once notified by the COR, the contractor shall send the materials to districts/schools and follow-up with the necessary telephone calls and/or site visits to secure participation in the evaluation.

5.2 Memoranda of Understanding (MOU): The contractor shall establish and sign a formal MOU with each participating school district. The MOU shall indicate the roles, responsibilities, and expectations of each party. The MOU shall include a specific agreement regarding when districts will make available interim attendance data so that the second randomization can be implemented. The MOU shall be signed by the contractor and the lead school district staff. The MOU shall include, but not be limited to, the following topics: the intervention; the proposed evaluation design and work plan; random assignment procedures; implementation of the intervention; data needs; compensation to offset financial burdens associated with participation in the evaluation. The contractor shall have MOUs signed no later than August 1, 2017.

As part of the recruitment process, the contractor shall comply with: The Privacy Act of 1974, P.L. 93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and Privacy Act of 1974, 20 USC 1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41 CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the Federal common rule or the Department’s final regulations on the protection of human research participants.

5.3 Recruitment Progress Reports: During recruitment (roughly May or June 2017-August 2017), the contractor shall submit a brief update to the COR that…