DRAFT_EOC_PWS_2.8.2016.pdf

DRAFT EOC PWS

DRAFT Performance Work Statement

STUDY OF AN INFORMATION STRATEGY TO INCREASE

ENROLLMENT IN POSTSECONDARY EDUCATION

U.S. Department of Education

I. INTRODUCTION

The U.S. Department of Education (ED) requires a study of an information strategy to increase participation in postsecondary education among individuals from low income families and those who are potential first generation college students. It focuses on the use of messaging targeted toward clients of Educational Opportunity Centers (EOCs) regarding college enrollment activities and deadlines. The EOC program is one of ED’s

TRIO programs.

The period of performance shall consist of a 19 month Base Period and a 56 month

Option Period. If the option period is exercised, the contract will have a total performance period of 75 months.

A. Legislative Authority

Section 402(H) of the Higher Education Opportunity Act of 2008 (20 U.S.C. 1070a–18) authorizes the Secretary to make grants or enter into contracts to rigorously evaluate the effectiveness of strategies and practices supported under TRIO programs.

B. Study Background

There has been long-standing public policy interest in increasing college enrollment and completion among disadvantaged individuals. Participation in postsecondary education is associated with numerous positive outcomes, including increased employment rates and higher salaries.

However, high school graduates from low income families are less likely to enroll in postsecondary education than graduates with higher incomes. In 2013, approximately 46% of students from low income families went immediately from high school on to two- or four-year colleges, while approximately 79% of students from high income families did the same.

U.S. Department of Labor, Bureau of Labor Statistics, Office of Employment and Unemployment

Statistics, unpublished annual average data from the Current Population Survey (CPS), selected years, 2000 through 2014. See Digest of Education Statistics 2014. http://nces.ed.gov/programs/coe/indicator_cbc.asp, accessed on August 21, 2015.

U.S. Department of Commerce, Census Bureau, Current Population Survey (CPS), March 1996 through

March 2014. https://nces.ed.gov/programs/digest/d14/tables/dt14_502.30.asp, accessed on August 19, 2015.

U.S. Department of Commerce, Census Bureau, Current Population Survey (CPS), October, 1975 through

2013. https://nces.ed.gov/programs/digest/d14/tables/dt14_302.30.asp, accessed on August 19, 2015.

http://nces.ed.gov/programs/coe/indicator_cbc.asp https://nces.ed.gov/programs/digest/d14/tables/dt14_502.30.asp https://nces.ed.gov/programs/digest/d14/tables/dt14_302.30.asp

The Federal TRIO Programs are a set of eight outreach and student services programs designed to improve the college preparation and success of individuals from disadvantaged backgrounds. Together, the programs provide assistance from as early as middle school through the academic pipeline into post-baccalaureate programs.

The TRIO program that is the focus of this study—the Educational Opportunity Centers

(EOCs) program— provides counseling and information on college admissions to adults who want to enter or continue a program of postsecondary education. Grantees—which are largely institutions of higher education—serve individuals who are 19 years and older, most of whom are potential first-generation college goers and who have low incomes.

The study is designed to examine a strategy which may improve EOCs’ effectiveness in meeting the program’s aim of increasing the number of disadvantaged adults who enroll in postsecondary education. EOC grantees provide information and counseling to clients in-person and via telephone. ED seeks to test whether systematic, low-cost messaging can enhance these services. Research has shown that carefully crafted messages incorporating behavioral science insights can be effective in prompting high school graduates to enroll in postsecondary education and college freshman to renew their Free

Application for Federal Student Aid (FAFSA) applications.

Based on this research, a hypothesis is that messages that include timely, personalized information concerning college enrollment activities and deadlines, tips for completing forms and other procedures, and a link back to the center to answer questions and provide further assistance could serve as a low-cost method for the centers to effectively encourage clients toward college enrollment. The messaging also has the potential to enhance

EOCs’ productivity, by freeing up grantee staff time to focus on other grant activities that cannot be accomplished through automated messaging.

C. Key Research Questions

The study will address the following research questions:

1. Does providing personalized messages to EOC clients increase FAFSA completion rates?

2. Do the messages increase the likelihood of EOC clients enrolling in postsecondary education?

D. Evaluation Approach

Castleman, B & Page, L. (2014). Summer Nudging: Can Personalized Text Messages and Peer Mentor

Outreach Increase College Going Among Low Income High School Graduates? EDPolicyWorks Working

Paper, retrieved June 18, 2015 from:

http://curry.virginia.edu/uploads/resourceLibrary/9_Castleman_SummerTextMessages.pdf; Castleman, B

& Page, L. (2014). Freshman Year Financial Aid Nudges: An Experiment to Increase FAFSA Renewal and

College Persistence. EDPolicyWorks Working Paper, retrieved June 18, 2015 from:

http://curry.virginia.edu/uploads/resourceLibrary/29_Freshman_Year_Financial_Aid_Nudges.pdf;

http://curry.virginia.edu/uploads/resourceLibrary/9_Castleman_SummerTextMessages.pdf http://curry.virginia.edu/uploads/resourceLibrary/29_Freshman_Year_Financial_Aid_Nudges.pdf

ED’s assumptions about the study are provided below.

Up to 20 EOC grantees will be recruited to participate in the study.

Across two grant years (beginning in spring 2018 and through spring 2020), a total of approximately 6,000 EOC clients will be recruited to participate in the study.

Clients eligible for the study will be those who are new to the grantee (i.e. have not received services from the grantee in the past) and who intend to enroll at a postsecondary institution by the Fall semester of 2020.

The contractor will randomly assign eligible clients within EOCs to a treatment group that receives the messaging or a control group that does not receive the messaging. Both the treatment and control groups will receive all regular EOC services to which they would normally have access.

The study shall involve the collection of the following types of data.

Client Information. Each Center’s existing client intake form(s) will be augmented as needed in order to: identify clients eligible for the study, collect background information such as demographics, collect information needed for the customization of messages, and obtain client contact information. Client consent to participate in the study will also be needed in order to provide messages to the client should s/he be assigned to the treatment group.

Implementation Information. From the messaging system, the contractor shall obtain implementation information such as the number of messages sent to and from each treatment client and the number of messages sent to/from each EOC staff member who has access to the system.

Federal Aid Application Receipt. Via a matching process conducted through the

Department’s Office of Financial Student Aid, the contractor shall determine whether study clients completed and submitted the FAFSA and the amount of their financial awards.

College Enrollment. Based on data obtained from the National Student

Clearinghouse, the contractor shall determine study clients’ college enrollment status as of Fall 2018, Fall 2019, or Fall 2020, depending on when the client had intended to enroll.

II. SCOPE OF WORK

ED anticipates that the Base Period of performance will begin in June 2016 and end at the end of November 2017. During the Base Period, the contractor shall perform Tasks 1 through 7 below.

Task 1—Communication with ED (Hybrid)

1.1 Kickoff Meeting (Fixed Price)

The contractor’s project director and key project staff shall meet with the Contracting

Officer’s Representative (COR) and other appropriate ED staff within 10 business days after the effective date of the contract award to discuss details regarding the tasks outlined in the proposal, study design, scheduling activities, and other issues related to the contract. This meeting shall be at ED's offices in Washington, DC. The contractor shall provide an agenda for the meeting at least 3 business days prior to the meeting. The

COR will review the agenda and provide comments at least 2 business days prior to the meeting. The contractor shall submit a final agenda at least 1 business day prior to the meeting. The contractor shall come prepared to identify any areas of concern and to suggest ways of responding to these concerns. Within one week following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each shall be addressed in the study.

1.2 Ongoing Communication (Cost Reimbursement)

The contractor shall have conference calls with the COR in order to discuss design decisions, study progress, and address any issues that may arise on the study. The calls shall be held on an as needed basis. For planning purposes, the contractor shall assume biweekly phone calls throughout the base period of the contract.

Deliverable: Draft meeting agenda

Final meeting agenda

Memo summarizing kickoff meeting

Task 2—Technical Working Group (Fixed Price)

2.1 Establish EOC Technical Working Group (TWG)

The role of the TWG will be to advise the contractor on the conduct of the study including, but not limited to: evaluation design, instrumentation, data collection and analysis, and draft evaluation reports. The TWG shall be comprised of up to eight individuals and should include experts in postsecondary education policies, behavioral and information sciences, and evaluation methodology. The contractor shall submit a list of proposed TWG members no later than one week after the kickoff meeting. When preparing the list, the contractor shall evaluate the potential for conflicts of interest by assessing whether each potential TWG nominee has a bias or financial interest in the outcome of the study. After the COR endorses the proposed TWG members, the contractor shall contact each proposed member and, within four weeks of COR endorsement, submit letters of commitment to the COR and Contracting Specialist (CS) for approval by the Contracting Officer (CO).

2.2 Convene First EOC Technical Working Group Meeting

The contractor shall convene one meeting of the TWG during the Base Period. The meeting shall be held no later than November 1, 2016 and shall focus on study and intervention design issues. The contractor shall arrange the meeting location to minimize the travel of ED staff. The contractor shall pay the travel, per diem, and honorarium expenses of the TWG members.

The contractor shall submit a written summary of the meeting to the COR within one week following the meeting, detailing suggestions and recommendations made by the

TWG members. The summary shall include any refinements to the study design that are proposed by the contractor as a result of TWG feedback. The contractor shall not make any technical changes without COR approval. The COR will review and provide comments on the summary within one week. If needed, the contractor shall submit a revised summary within one week of receiving comments from the COR.

In addition to the TWG meeting, the contractor also shall use the TWG members to review products, as appropriate, when group meetings are not required. The contractor shall provide copies of all reviews of products by the panel to the COR within five work days of receipt by the contractor. If TWG members recommend any significant deviation from the proposal, study design, instrumentation, data collection, or data analysis, the contractor shall submit the recommendation to the COR. The COR, in consultation with the CO, will assess whether the changes warrant a change to the contract. The contractor shall not make any significant changes in the study based solely upon the recommendation of TWG members

Deliverables: List of proposed TWG members

Letters of commitment

Summary of first TWG meeting

Revised summary of first TWG meeting

Task 3 - Recruitment of EOC Grantees (Cost Reimbursement)

The contractor, working with the COR and in consultation with the TRIO community, shall develop a strategy for identifying and recruiting EOC grantees that is appropriate for effectively and efficiently addressing the study’s research questions. The strategy should take into consideration the number of clients served by the grantee, the proportion of those who may intend to enroll in college during the target period, and the geographic representation among the selected grantee sample.

As part of the recruitment process, the contractor shall prepare notification letters and information packets for the prospective study grantees. The notification packet shall include a non-technical brochure describing the study that is suitable for distribution to a broad audience of policy makers and educators. The contractor shall include in the letters and information packets general information on the study as well as specific information on the importance of the study, its purposes, products, timelines, and the expected benefits to be derived from participation in the study. The contractor shall submit draft letters and all other notification materials to the COR for review by November 15, 2016.

Once approved by the COR, the contractor shall send the notification materials to grantees and follow-up with the necessary telephone calls to secure the cooperation of grantees in the evaluation.

The contractor shall also host up to two webinars to provide information regarding the study to interested grantees. The webinars, which the COR anticipates will be identical in content, shall be scheduled to maximize grantee participation and shall highlight the requirements of the study as well as the benefits to grantees. The contractor shall submit draft webinar slides to the COR two weeks prior to the first scheduled webinar and revised webinar slides at least one week prior to the first scheduled webinar date.

As part of the recruitment process, the contractor may also make a presentation to grantees during a meeting of the Council for Opportunity in Education.

The contractor shall consult with the COR to finalize the list of grantees for the evaluation, and subsequently submit memoranda of understanding for those grantees that will participate in the evaluation to the COR by July 1, 2017.

As part of the recruitment process, the contractor shall comply with: The Privacy Act of

1974, P.L. 93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and

Privacy Act of 1974, 20 USC 1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41 CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the Federal common rule or ED’s final regulations on the protection of human research participants.

During recruitment, the contractor shall submit to the COR progress reports on grantee recruitment for the study on a biweekly basis.

Deliverables: Grantee recruitment materials

Draft webinar slides

Final webinar slides

Memoranda of understanding from grantees

Progress reports on grantee recruitment

Task 4 – Obtain Office of Management and Budget (OMB) and Institutional Review

Board (IRB) Clearance (Cost Reimbursement)

The contractor shall prepare all relevant forms and justification for a data clearance package, including data collection instrumentation. The contractor shall ensure that the clearance package justifies the study’s data collection activities and comprehensively responds to each required item in the OMB instructions. In addition, the contractor shall ensure that the forms clearance package includes: (1) the study mandate and objectives,

(2) the research questions, (3) description of and justification for incentives, (4) description of sampling and recruiting procedures, and (4) limitations of the study and its potential use for policy decisions.

The contractor shall design data collection instruments to address the key evaluation questions. The instruments shall encompass the data collections described within Section

D of this Performance Work Statement. They shall include a set of data items to be integrated (under Task 6) with study grantees’ existing client intake forms. This item set shall be designed to: determine client eligibility for the study, collect descriptive and contact information from study-eligible clients, identify postsecondary institutions in which study-eligible clients intend to enroll, and obtain the information necessary for the study team to obtain Federal Student Aid (FSA) data and college enrollment data, under

Task 11. The contractor shall make use of or refine existing instruments to the extent possible and applicable.

The contractor shall submit a draft clearance package to the COR by November 30, 2016.

The COR will provide comments on the draft package within 1 week. The contractor shall submit a revised package to the COR by December 15, 2016. Once the COR has approved the revised package, the COR will then submit the package to ED’s Office of the Chief Privacy Officers, Information Collection Clearance Division (ICCD). ICCD will submit the package to OMB for review. The contractor shall revise the package based upon OMB comments, as needed, and submit the final package to the COR within one week of receiving comments.

The contractor shall submit the OMB forms mindful that clearance requires a minimum of 120 days and how this corresponds with the agreed schedule of deliverables.

The contractor shall pilot test each of the instruments with no more than 9 respondents per instrument to ensure that the questions can be answered in a reasonable amount of time, that the directions are easily followed, and that the contractor’s procedures and testing systems for administering the instruments are effective. Piloting shall be conducted by the contractor within the first public comment period, prior to the final 60 days in the OMB submission process. The contractor shall ensure that instruments are revised as needed based on pilot test results.

The contractor shall coordinate its own Institutional Review Board (IRB) approval for the piloting, field testing, and full scale data collection with any IRB approval necessary by grantee institutions. Specifically, the contractor is responsible for ensuring that all aspects of the program have been approved by the appropriate IRBs, including its own

IRB, and of notifying ED that this has been accomplished.

Deliverables: Draft clearance package with instruments

Revised clearance package with instruments

Final clearance package with instruments

Notification of IRB approval

Task 5 – Design of the Intervention (Cost Reimbursement)

The contractor shall design the intervention, bearing in mind that EOC clients may or may not have applied to a postsecondary institution at the time of program intake and random assignment.

The contractor shall develop the messages to be sent to treatment clients. The messages shall pertain to college affordability and matriculation, including financial aid award letters, college fees, orientation and registration deadlines, and college enrollment requirements.

The COR assumes there will be no more than three messages sent per client each month.

The content, package, and delivery of this information shall be designed based on research in behavioral economics.

Some messages (such as some of those regarding the

FAFSA and the Common Application) shall be uniform across clients while others will require customization based on the client’s intended postsecondary institution.

By April 1, 2017, the contractor shall submit to the COR a proposed intervention plan.

The proposed plan shall include draft messages (including message shells for those that will require customization) and a strategy for implementing the messages with EOC clients who may be at different stages of applying and enrolling in postsecondary education programs. The COR will review and provide comments within two weeks of receipt. The contractor shall submit a final intervention plan by June 1, 2017.

Deliverables: Proposed Intervention Plan

Final Intervention Plan

Task 6 – Design of EOC Client Recruitment and Random Assignment Process (Cost

Reimbursement)

6.1 Collaborate with Study Grantees to Amend Intake Forms

Following receipt of OMB approval for data collection, the contractor shall work with study grantees to incorporate necessary baseline items into grantees’ existing client intake forms, if they are not already included. Depending on grantees’ existing practices, the intake forms may be forms that clients complete in hard copy, online forms, and/or forms that are completed by EOC staff over the phone with clients.

6.2 Design Random Assignment Procedures

The contractor shall design procedures such that eligible and consenting clients are randomly assigned within grantee to the treatment or control condition. Depending on the grantees included in the study and their client intake process, there may be a set of common random assignment procedures across all study grantees or the contractor may need to design multiple sets of procedures to accommodate grantees. By September 15, 2017, the contractor shall send the COR a memo that describes how the random assignment process will be carried out across the grantee sites.

6.3 Design Grantee Intake Training

For instance, see Lavecchia, A.M., Liu, H., & Oreopoulos, P. (October, 2014). Behavioral Economics of

Education: Progress and Possibilities. National Bureau of Economic Research, Working Paper Series.

Retrieved June 10, 2015 from http://www.nber.org/papers/w20609.pdf.

http://www.nber.org/papers/w20609.pdf

The contractor shall design training for grantee intake staff on study procedures, including determining client eligibility for the study, the consent process, and collecting all study-related information on the intake form. The contractor shall send the COR the draft training materials, including slides and handouts, if any, by October 15, 2017. The

COR will provide comments within one week. The contractor shall provide revised training materials by November 1, 2017.

Deliverables: Random Assignment Process Memo

Draft Grantee Intake Training Materials

Revised Grantee Intake Training Materials

Task 7 -- Reporting Requirements, Months 1 through 19 (Fixed Price)

The contractor shall submit one copy of the following performance reports each month to the CS, CO, and one copy to the COR simultaneously with the monthly invoice.

Monthly Progress Report/Exception Reports. The contractor shall prepare monthly progress reports due at the same time as the monthly invoice. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from the COR. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

Monthly Manpower/Expenditure Reports. The contractor shall prepare monthly expenditure reports due at the same time as the monthly invoice. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Deliverables: Monthly progress reports

Optional Period I

ED anticipates that the Optional Period, if exercised, will begin in December 2017 and end at the end of July 2022. During this period, the contractor shall perform Tasks 8 through 13 below.

Task 8 - Ongoing Communication (Cost Reimbursement)

The contractor shall have conference calls with the COR in order to discuss design decisions, study progress, and address any issues that may arise on the study. The calls shall be held on an as needed basis. For planning purposes, the contractor shall assume biweekly phone calls throughout the option period of the contract.

Deliverables: None

Task 9 – Obtain Intake Data from EOCs and Randomly Assign Eligible Clients

(Cost Reimbursement)

9.1 Train Grantee Intake Staff

Using the materials developed under Task 6, the contractor shall train grantee intake staff on study intake and random assignment procedures. For planning purposes, the contractor shall assume that the training will be delivered via webinar with up to five webinars required to accommodate grantee.

9.2 Randomly Assign Clients

Beginning in approximately February of 2018, the contractor shall obtain from the study grantees intake form data for eligible and consenting clients. These data shall be collected on a rolling basis through the spring of 2020. The contractor shall randomly assign eligible clients within EOCs to the treatment group or the control group and monitor implementation to ensure adherence to the random assignment. The contractor shall design a process for checking on random assignment and intake to the study sample.

The contractor shall implement the process and work with grantees to resolve any issues identified. The contractor shall provide monthly updates to the COR on the study sample size within each EOC.

Deliverables: Monthly Sample Size Updates

Task 10 – Implementation of the Intervention (Cost Reimbursement)

The contractor shall hold an open competition and select a messaging service provider to send the messages to members of the treatment group. The provider’s platform shall be flexible enough to allow for multiple approaches to client messaging. The platform shall also allow for two-way communication such that clients and EOC staff can both receive and respond to the messages.

The contractor shall implement the messaging system with the clients assigned to the treatment group, using the information collected through the client intake forms (Task 9).

The messages shall be delivered to clients on a schedule such that they coincide with upcoming enrollment deadlines and events. The contractor shall work through grantees to obtain information specific to postsecondary education institutions, in order to inform the timing and content of the messages.

ED anticipates that the messages will be sent out between spring 2018 and summer 2020, targeted toward clients who are intending to enroll in college between the point of random assignment and fall 2020.

Client responses to messages shall be accessible to study grantee staff. Grantee staff will be responsible for following up with clients as needed. To facilitate this, prior to the start of messages being sent to clients, the contractor shall provide webinar training to grantee staff on how to access the messaging system and retrieve messages from students. The contractor shall also provide followup technical assistance to grantee staff on an as needed basis, to troubleshoot any logistical or technical issues that arise as staff attempt to access the system. The contractor shall provide draft webinar training slides to the

COR at least two weeks prior to the webinar and final webinar slides at least one week in advance of the training.

Deliverables: Draft messaging training materials

Final messaging training materials

Task 11—Data Collection (Cost Reimbursement)

From the messaging system, the contractor shall obtain implementation information such as the number of messages sent to and from study clients and EOC staff.

The contractor shall obtain data to examine impacts on federal financial aid application

(FAFSA) and receipt. The contractor shall obtain data from FSA systems, working through the COR. The contractor shall prepare data requests to FSA specifying the clients for whom information is being sought and the fields/variables to be extracted. The contractor shall submit the first request to the COR by December 2018. The request shall pertain to clients who had intended to enroll at a postsecondary education institution by

Fall 2018. The contractor shall submit a second and final request to the COR by

December 2020. This request shall pertain to clients who had intended to enroll at a postsecondary education institution between Spring 2019 and Fall 2020. The contractor shall be prepared to answer any questions that FSA may have in carrying out the extraction of the data.

The contractor shall obtain data on study clients’ college enrollment outcomes directly from the National Student Clearinghouse. These data shall be obtained in December

2018, December 2019, and December 2020, with each collection including students who had intended to enroll at a postsecondary education institution within the ending calendar year.

The contractor, and all subcontractors, shall comply with: The Privacy Act of 1974, P.L.

93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and Privacy Act of 1974, 20 USC 1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41 CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the Federal common rule or ED’s final regulations on the protection of human research participants.

The contractor shall be required to establish procedures at its sites and at subcontractor sites to ensure confidentiality of data. Their systems shall be required to provide reasonable assurance that information identifying individuals is in files physically separated from other research data. The contractor and subcontractors shall be required to maintain security of the complete set of all master data files and documentation.

Access to individually identifiable data shall be strictly controlled. At each site all data shall be kept in locked file cabinets during nonworking hours, and work on hardcopy data shall take place in a single room, except for data entry. Physical security of electronic data shall also be maintained. Security features that protect project data shall include password-protected accounts that authorize users to use the contractor’s and subcontractors’ information systems but only to access specific network directories and network software; user rights and directory and file attributes that limit those who can use particular directories and files and determine how they can use them; and, additional security features that the network administrators establish for projects as needed.

Contractor and subcontractor employees who “maintain” (collect, maintain, use, or disseminate) such data shall comply with the requirements of the confidentiality standards in section 183 of the ESRA (20 U.S.C. 9573).

In safeguarding personally identifiable information (PII), the contractor and its subcontractors also shall be subject to ED’s requirements contained in ED’s Handbook for the Protection of Sensitive But Unclassified Information, OCIO-15, and ED’s policy that the transmission of sensitive but unclassified information, including PII, through an e-mail requires that the contents be password protected in a ZIP file.

All persons who collect, code, or otherwise have access to raw data must be trained in confidentiality issues and shall provide signed assurances of nondisclosure of confidential information. The contractor shall maintain these records and provide them to the

Government upon request. When possible, the contractor shall ensure that individuals collecting and/or coding data do not have knowledge of the study participant’s treatment condition.

In accord with OMB Memorandum M-05-04, and with the NIST SP 800-44, all publicly accessible Federal websites and web services must have HTTPS enabled, and shall only provide service over a secure connection, and e-mail applications must have SMTP enabled.

In order to comply with the Federal Records Act of 1950, as amended (44 U.S.C. 3101-

3107), the Contractor shall certify to the COR that each staff member has completed the

ED's basic records management training within 10 business days of starting work on the contract.

Compliance with ED IT Security Policy

The contractor, and all sub-contractors, shall comply with the Department of Education’s

IT security policy requirements, specifically those set forth in the ‘Handbook for

Information Assurance Security Policy (OCIO-01)’, and other applicable procedures and guidance. The contractor, and all sub-contractors, shall develop and implement management, operational and technical security controls to assure required levels of protection for information systems. The contractor, and all sub-contractors, shall further comply with all applicable Federal IT security requirements including, but not limited to, the Federal Information Security Management Act (FISMA) of 2002, Office of

Management and Budget (OMB) Circular A-130 Appendix III, Homeland Security

Presidential Directives (HSPD), the National Institute of Standards and Technology

(NIST) standards and guidance, and the Federal Risk and Authorization Management

Program (FedRAMP) requirements and guidance.

These security requirements include, but are not limited to, the successful Security

Authorization (SA) of the system (includes commercially owned and operated systems managed by the commercial vendor and its sub-contractors, supporting Department programs, contracts, and projects); obtaining a full Authority to Operate (ATO) before being granted operational status; performance of annual self-assessments of security controls; annual Contingency Plan testing; performance of periodic vulnerability scans;

updating all information system security documentation as changes occur; and other continuous monitoring activities, which may include, mapping, penetration and other intrusive scanning. Full and unfettered access for the Department’s third party Managed

Security Services Provider (MSSP) must be granted to access all computers and networks used for this system. Additionally, when there is a significant change to the system’s security posture, the system (Federal and commercial prime- and sub- contractors included) must have a new SA, with all required activities to obtain a new ATO, signed by the Authorizing Official (AO).

System security controls shall be designed and implemented consistent with NIST SP

800-53 Rev 3, ‘Recommended Security Controls for Federal Information Systems and

Organizations.’ All NIST SP 800-53 controls must be tested / assessed no less than every

3 years, according to federal and Department policy. The risk impact level of the system will be determined via the completion of the Department's inventory form and shall meet the accurate depiction of security categorization as outlined in Federal Information

Publishing Standards (FIPS) 199, ‘Standards for Security Categorization of Federal

Information and Information Systems.’

System security documentation shall be developed to record and support the implementation of the security controls for the system. This documentation shall be maintained for the life of the system. The contractor, and all sub-contractors, shall review and update the system security documentation at least annually and after significant changes to the system, to ensure the relevance and accurate depiction of the implemented system controls and to reflect changes to the system and its environment of operation. Security documentation must be developed in accordance with the NIST 800 series and Department of Education policy and guidance.

The contractor, and all sub-contractors, shall allow Department employees (or

Department designated third party contractors) access to the hosting facility to conduct

SA activities to include control reviews in accordance with NIST SP 800-53, Rev. 3 and

NIST SP 800-53A. The contractor, and all sub-contractors, shall be available for interviews and demonstrations of security control compliance to support the SA process and continuous monitoring of system security. In addition, if the system is rated as

‘Moderate’ or ‘High’ for FIPS 199 risk impact, vulnerability scanning and penetration testing shall be performed on the hosting facility and application as part of the SA process. Appropriate access agreements will be reviewed and signed before any scanning or testing occurs.

Identified deficiencies between required NIST SP 800-53 Rev. 4 controls and the contractor’s, and all sub-contractor’s implementation, as documented in the Risk

Assessment Report, System Security Plan (SSP) and Security Assessment Report (SAR), shall be tracked for mitigation through the development of a Plan of Action and

Milestones (POA&M) in accordance with the ‘Handbook for Information Assurance

Security Policy (OCIO-01).’ Depending on the severity of the deficiencies, the

Department may require remediation before an ATO is issued.

All awarded contracts shall ensure that:

1. Their IT product/system is monitored during all hours of operations using entrusted detective/preventive systems;

2. Their IT product/system has current antiviral products installed and operational;

3. Their IT product/system is scanned on a reoccurring basis;

4. Vulnerabilities are remediated in a timely manner on their IT product/system; and

5. Access/view for cyber security situational awareness on their IT product/system is made available to the Department CIRC (cyber incident response capability).

Internet Protocol version 6 (IPv6)

For IPv6, the contractor shall provide COTS solutions that are IPv6 capable. An IPv6 capable system or product shall be capable of receiving, processing, transmitting and forwarding IPv6 packets and/or interfacing with other systems and protocols in a manner similar to that of IPv4. Specific criteria to be deemed IPv6 capable are:

An IPv6 capable system that meets the IPv6 base requirements defined by the

USGv6 Profile (http://www.antd.nist.gov/usgv6/profile.html).

Systems being developed, procured or acquired shall maintain interoperability with IPv4 systems/capabilities.

Systems shall implement IPv4/IPv6 dual-stack and shall also be built to determine which protocol layer to use depending on the destination host it is attempting to http://www.antd.nist.gov/usgv6/profile.html communicate with or establish a socket with. If either protocol is possible, systems shall employ IPv6.

The contractor shall provide IPv6 technical support for system development, implementation and management. System Development Standards: Information systems shall be developed in accordance with the ED Lifecycle Management Framework (LCM), ACS-OCIO 1-10.

Reporting of Data Security Breaches

If there is a suspected or known breach/disclosure of PII due to lost, theft, intercepted transfer, or other, the contractor must ensure that this breach is reported to the agency as soon as the contractor has knowledge of it. Per Office of Management and Budget

Memorandum M-06-19, Federal agencies have a requirement to report breaches of PII security to a Federal incident response center. IES must notify the department within 30 minutes of discovering the incident (and the agency should not distinguish between suspected or confirmed breaches). The data security plan must be written to reflect this requirement, and the contractor must provide sufficient notification and documentation of the suspected loss, as it is understood at the time of notification to the agency for this requirement to be met. Follow-up reports of the final status of loss events will also be prepared by the contractor within a reasonable period of time as advised by the IES COR.

Deliverables: Data Requests for FSA

Task 12 - Convene Second Technical Work Group Meeting (Fixed Price)

The contractor, in conjunction with ED, will determine the timing for the second TWG meeting. For planning purposes, the contractor shall assume the meeting will occur in

July 2021. This meeting will focus on preliminary study findings. As with the first TWG meeting (Task 2.2), the meeting will include up to eight TWG members. The contractor shall arrange the meeting location to minimize the travel of ED staff. The contractor shall pay the travel, per diem, and honorarium expenses of the TWG members.

The contractor shall submit a written summary of the meeting to the COR within one week following the meeting, detailing suggestions and recommendations along with the contractor’s proposed actions. The contractor shall not take action on TWG recommendations without COR approval.

Deliverables: Summary of 2 nd

EOC Study TWG meeting

Task 13—Data Analysis, Report Preparation, and Restricted Use Files (Cost

Reimbursement)

The contractor shall conduct data analyses and prepare one report that clearly addresses the key evaluation questions. The contractor shall ensure that the report is consistent with NCES and IES peer review standards. The contractor shall also develop a second short report that describes the intervention.

13.1 Data Analysis Plan

The contractor shall prepare a detailed data analysis plan that documents the intended analyses to be conducted and included in the report. The contractor shall ensure that the data analysis plan documents regression analyses (including a list of dependent and independent variables), proposed sensitivity analyses, proposed subgroup analyses, and table shells for potential inclusion in the report. The contractor shall submit the draft data analysis plan by August 1, 2020. The COR will review and provide comments within 3 weeks. The contractor shall revise based on COR comments and submitted a revised plan by September 15, 2020.

13.2 Report

The contractor shall ensure that the report is written and organized so that it is accessible to policy makers and research-savvy practitioners rather than for academic researchers.

The contractor shall include a stand-alone executive summary of the study’s findings.

ED anticipates that the report will be a maximum of fifty pages, excluding the executive summary and technical appendices.

The contractor shall submit a draft report no later than January 15, 2022. The COR will review and provide comments within three weeks. The contractor shall submit a revised draft report ready for SRO by March 1, 2022. ED will submit to SRO. Within approximately 2 months, ED will provide comments to the contractor from SRO. The contractor shall make final revisions based on SRO comments. The contractor shall revise the report until all peer review comments are addressed and the report clears SRO

(current experience is an average of 3 rounds of successively less extensive comments).

The contractor shall provide a web-ready version of the report to the COR within 2 weeks after the report is cleared by SRO. The web-ready version shall be a PDF file that is 508 compliant.

13.3 Intervention Description

The contractor shall develop a short report that describes the intervention, including the steps to implement it in the field. This report will be for the purpose of providing information to EOC grantees and others who may be interested in adopting the intervention, should the study reveal evidence of its effectiveness. ED anticipates that this report will be a maximum of twenty pages. The contractor shall submit a draft of the intervention description report to the COR by March 15, 2022. The COR will provide comments within two weeks. The contractor shall submit a revised intervention description report to the COR by April 15, 2022.

13.4 Data Files

The contractor shall prepare a set of restricted use data files. The files shall contain all data collected for the evaluation with all direct personal identifiers removed. The contractor shall provide documentation for the files including variable labels, category labels, weights, and how missing data were addressed. In addition, the contractor shall submit documentation accompanying the data to include the unit of observation for each data file (e.g., client, center, etc.).

The contractor shall also provide IES with a risk disclosure analysis for submission to

NCES’ Disclosure Review Board. Once the codebook and risk disclosure analysis are approved, the contractor shall submit to the COR the restricted use data files consistent with NCES standards . ASCII or SAS file formats are acceptable. The contractor shall submit the restricted use files by June 15, 2022.

During the course of the study and at its completion, the contractor shall be prepared to present several, not to exceed four, briefings of the results for program staff and/or policy makers if asked to do so.

Deliverables: Draft Analysis Plan

Revised Analysis Plan

Draft Report

Revised Report, SRO-ready

Web-ready Report

Draft Intervention Description Report

Revised Intervention Description Report

Restricted Use Data File with Documentation

Task 14 -- Reporting Requirements, Months 20 through 75 (Fixed Price)

The contractor shall submit one copy of the following performance reports each month to the contracting officer (CO) and one copy to the COR simultaneously with the monthly invoice.

Monthly Progress Report/Exception Reports. The contractor shall prepare monthly progress reports due at the same time as the monthly invoice. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

http://nces.ed.gov/statprog/style.asp

Monthly Manpower/Expenditure Reports. The contractor shall prepare monthly expenditure reports due at the same time as the monthly invoice. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Deliverables: Monthly progress reports

III. SCHEDULE OF DELIVERABLES

The contractor shall submit one copy of each final deliverable to the CO; all other deliverables (to include drafts) must be submitted to the COR. The contractor shall submit an electronic copy of all deliverables to the CO and COR unless otherwise specified. Deliverables shall be submitted by the due dates listed in the chart below unless otherwise noted in individual task descriptions. Deliverable review timeframes are noted in the individual task descriptions.

Deliverables

Due Date

Task 1 –Communication with ED

Draft meeting agenda At least 3 business days prior to meeting

Final meeting agenda At least 1 business day prior to meeting

Memo summarizing meeting 1 week after meeting

Task 2- Technical Work Group

List of proposed TWG members 1 week after kickoff meeting

Letters of Commitment 4 weeks after IES endorsement

Summary of first TWG meeting 1 week after meeting

Revised summary of first TWG meeting 1 week after receiving comments from the

COR

Task 3 – Recruitment of EOC Grantees

Draft recruitment materials November 15, 2016

Draft webinar slides Two weeks prior to first webinar

Final webinar slides One week prior to first webinar

Memoranda of understanding from grantees July 1, 2017

Progress reports on grantee recruitment Bi-weekly during recruitment

Task 4 – Obtain OMB and IRB Approval

Draft clearance package November 30, 2016

Revised clearance package December 15, 2016

Final clearance package Within one week of receiving comments from OMB

Notification of IRB approval

Task 5 – Design of the EOC Intervention

Proposed Intervention Plan April 1, 2017

Final Intervention Plan June 1, 2017

Task 6 – Design of the EOC Client Recruitment and Random Assignment Process

Random Assignment Process Memo September 15, 2017

Draft Grantee Intake Training Materials October 15, 2017

Revised Grantee Intake Training Materials November 1, 2017

Task 7 - Reporting Requirements, Month 1 - 19

Monthly reports Submitted simultaneously with monthly invoice

EOC Study Option Period I

Task 8 – Ongoing Communication

None

Task 9 – Obtain Intake Data from EOCS and Randomly Assign Eligible Clients

Sample Size Updates Monthly during the rolling random assignment period

Task 10 – Implementation of the Intervention

Draft Messaging Training Materials 2 weeks prior to training

Final Messaging Training Materials 1 week prior to training

Task 11 –Data Collection

Data Requests for FSA December 2018

December 2020

Task 12 – Convene Second EOC TWG Meeting

Summary of 2 nd

EOC Study TWG meeting 1 week after meeting

Task 13 – Data Analysis and Report Preparation

Draft Analysis Plan August 1, 2020

Revised Analysis Plan September 15, 2020

Draft Report January 15, 2022

Report, SRO ready March 1, 2022

Web-ready Report Within 2 weeks after

SRO approval

Draft Intervention Description Report March 15, 2022

Revised Intervention Description Report April 15,…