TSL_-_PWS.docx

U.S. Department of Education Performance Work Statement Institute of Education Sciences National Center for Education Evaluation

Performance Work Statement

An Impact Evaluation to Inform the Teacher and School Leader Incentive Program

INTRODUCTION

The National Center for Education Evaluation and Regional Assistance (“NCEE”) of the U.S. Department of Education’s (the “Department”) Institute of Education Sciences (IES) requires a contract to conduct a two-component study. The first component includes collecting implementation information about the Teacher and School Leader Incentive Fund (TSL) program FY2017 grants. The second component includes an impact evaluation of a core strategy that is common among the funded TSL FY2017 grants. In addition, the contractor shall collect data to provide information for the program’s performance indicators.

This contract includes a 27-month base period and three optional task periods that would result in a 59-month contract in total. The base contract includes collecting in-depth information about the teacher leader role to inform the characteristics of the intervention to be studied in the impact evaluation component of the study; study refinement, recruitment of study schools to participate in the impact evaluation as well as the development of data collection forms and development of the OMB forms clearance package, a TWG meeting (with design memo); and recruitment of teacher leaders and initial implementation of the impact evaluation component. Based on documented successful conduct of Base period study activities, the Department will decide whether to exercise Optional Task Period I by September 2019.

Optional Task Period I includes a grantee survey in spring 2020 to collect information about TSL implementation and GPRA data requirements, implementation of the teacher leader role strategy within the context of the impact evaluation component, and data collection during the 2020-21 school year for both the implementation and impact components. Optional Task Period I is 28 months and overlaps with the base contract by 15 months. Based on documented successful conduct of Optional Task Period I study activities, the Department will decide whether to exercise Optional Task Period II by May 2020.

Optional Task Period II includes a memo to provide the program office with information about the TSL GPRA measures and initial implementation information (based on spring 2020 grantee survey data), data collection including administrative data and the spring 2021 grantee survey administration, data analyses, a technical working group meeting, and initial draft report writing. Optional Task Period II is 25 months and overlaps with the Optional Task Period I by 21 months. Based on documented successful conduct of Optional Task Period II study activities, the Department will decide whether to exercise Optional Task Period III by March 2021.

Optional Task Period III is 30 months and overlaps with the Optional Task Period II by 15 months. Optional Task Period III includes a GPRA memo (based on grantee survey data from 2021), and final draft report preparation and revisions through IES’s standards and review office (SRO) clearance process.

A. Legislative Authority

TSL is authorized in the Elementary and Secondary Education Act of 1965 as amended through P.L. 115-64. The Director of IES is directed to conduct an independent evaluation to measure the effectiveness of the program. Further, the evaluation must “measure the effectiveness of the program in improving student academic achievement, the satisfaction of the participating teachers, principals, or other school leaders, and the extent to which the program assisted the eligible entities in recruiting and retaining high-quality teachers, principals, or other school leaders, especially in high-need subject areas.”

B. TSL Program The TSL program assists States, local educational agencies (LEAs), and nonprofit organizations to develop, implement, improve, or expand comprehensive performance-based compensation systems or human capital management systems for teachers, principals, and other school leaders (especially for teachers, principals, and other school leaders in high-need schools) who raise student academic achievement and close the achievement gap between high- and low-performing students. The FY2017 notice inviting applications contained four absolute priorities and two competitive preference priorities (see federal notice, https://www.gpo.gov/fdsys/pkg/FR-2016-12-20/pdf/2016-30643.pdf).

A major objective of the program is to improve educator effectiveness through the use of human capital management systems that measure performance and guide ongoing, targeted support. The Every Student Succeeds Act (ESSA) includes a mandated independent evaluation of the program that measures the effectiveness of the program in:

· Improving student academic achievement;

· The satisfaction of participating teachers, principals, or other school leaders; and

· Improving recruiting and retaining high-quality teachers, principals, or other school leaders.

C. Key Research Questions

One goal of this evaluation is to provide implementation information about the activities funded through the FY2017 TSL grants. A second goal is to study the effectiveness of a commonly funded component of TSL grants. Because the grantees have already been awarded, the impact evaluation aspect of the study will be conducted in non-TSL grantee districts or schools. The evaluation report will include both a description of TSL grantee schools and activities as well as effects of the commonly funded component. The report shall link the impact evaluation results as much as possible to TSL funded activities for the purpose of program improvement.

A review of TSL grants awarded in FY2017 indicates that the grants include multiple strategies across a wide array of strategies, with the most common strategy being funding for teacher leadership roles. The teacher leadership role typically offers additional compensation to highly effective teachers to take on a leadership role that includes working with their lower-performing colleagues to improve teaching. The evaluation shall include both impact and implementation information to address the following research questions.

Implementation evaluation component (TSL grantees):

1. What are the implementation experiences of FY2017 TSL grantees? What are their educator satisfaction, recruitment, and retention experiences with TSL, particularly among those grantees funding teacher leader roles?

Impact evaluation component (non-TSL sites):

2. What are the experiences of sites implementing a teacher leader role?

3. What is the effect on student achievement of funding teacher leader role strategy? What is the impact of a teacher leader role strategy on educator satisfaction, recruitment, and retention?

4. What is the cost effectiveness of the teacher leader role strategy?

Overarching lessons from the two-component study:

5. How do the two evaluation components inform the TSL program, in terms of experiences and challenges of districts and educators when implementing the teacher leader role strategy? In terms of effects?

The impact evaluation component shall include voluntary districts (not TSL grantees) willing to implement a teacher leader role strategy and participate in an impact evaluation. To make the evaluation results most relevant to the FY2017 TSL grants, the contractor shall recruit schools similar in characteristics to TSL grantee schools that agree to implement the teacher leader strategy. In addition, the specific teacher leader strategy studied in the non-TSL schools shall be similar in nature to that typically implemented by TSL grantees. The evaluation shall include data collection of TSL grantees as well as within districts and schools participating in the impact evaluation, as indicated in Task 2b.

D. Evaluation Approach and Implementation of Teacher Leader Role

The purpose of the Department’s evaluation is to evaluate the effectiveness and cost effectiveness of teacher leader roles similar to those funded by the TSL program and to describe TSL grants, both generally and specifically related to implementation of the teacher leader role. The contractor shall consider the following assumptions:

· The Department awarded 14 FY2017 TSL grants (Nine of which proposed a teacher leader role strategy: see https://innovation.ed.gov/what-we-do/teacher-quality/teacher-and-school-leader-incentive-program/awards/). Implementation and GPRA data shall be collected in all 14 grants. Teacher leader role-specific data shall be collected from the 9 grants.

· The impact study shall include approximately 10 districts and 100 schools. Recruited schools shall be high-need, consistent with eligibility focus for TSL grants.

· Within each of the participating districts, schools shall be randomly assigned to treatment (high-performing teachers within each assigned treatment school shall be identified and recruited by the contractor to receive payment in exchange for providing support to 2-3 low-performing teachers in their school) or control (these schools do not use teacher leaders to support their low-performing teachers).

· The contractor shall provide teacher leaders with an orientation session prior to the start of the 2020-21 school year that includes expectations about their teacher leader role. In addition, the contractor shall provide implementation support consistent with how TSL grant implementation.

· Up to two teacher leaders per school.

Data collection for the impact evaluation component shall consist of teacher leader enrollment and activity forms, a teacher survey for the implementation year (background, support experiences and professional development activities and hours), and a principal survey, student records data for students of the teacher leaders and the teachers in the grade in which they serve as a teacher leader, and a grantee survey (to broadly describe the strategies funded by TSL and to provide more in-depth information about teacher leader role activities and challenges for those grantees implementing the strategy). The contractor shall administer the grantee survey in spring 2020 and spring 2021. The grantee survey data collection shall also include data relevant for the program’s performance indicators, see https://www.gpo.gov/fdsys/pkg/FR-2016-12-20/pdf/FR-2016-12-20.pdf, p. 92804). To fully address the study’s mandate, relevant questions from the teacher and principal surveys shall be administered within a sample of the FY2017 TSL grantee schools (up to 100 principals and approximately 200 teachers in TSL schools).

E. Design Issues

Discussed below are a number of factors that will affect the evaluation design, and successful execution of the study and presentation of study findings. The contractor shall be cognizant of these issues while carrying out the evaluation.

· District and School Context. Districts with a context similar to that required in TSL are most applicable for this evaluation. Schools that meet TSL eligibility requirements are also most applicable for this evaluation. Thus, the contactor shall address in the study design and site recruitment aspects of the proposal.

· Counterfactual. Schools are likely to use multiple strategies to improve teacher performance. Thus, the evaluation design and data collection shall address how this complicates the study service contrast and thus the design and interpretation of the impact evaluation.

· Method of Identifying Teacher Leaders. The qualifications and skills of the recruited teacher leaders are likely to be a key factor in interpreting the impact results and replicating the strategy. The specific method used by TSL grantees is also important in making the results relevant to the TSL program effectiveness. Thus, the contractor shall address in the study design and site recruitment aspects of the proposal.

· Teachers Targeted for Support. Who is supported and how are also likely to be key factors in using the impact results to inform the TSL program. Thus, the contractor shall address in the study design and site recruitment aspects of the proposal.

· Survey Sample. The contractor shall collect relevant data for both impact study participants (teachers and principals) as well as TSL grantees to meet the study goals while minimizing data collection costs.

· Power analyses. The contractor shall calculate the appropriate sample size to achieve the targeted effect size of 0.10-0.12. The contractor shall fully explain and justify clustering as well as other power calculation assumptions.

SCOPE OF WORK

This section specifies the tasks and subtasks that the contractor shall perform.

Task 1—Opening Meeting with the Department

The contractor’s project director and two additional key project staff members shall meet with the COR and other appropriate Department staff within ten (10) work days after the effective date of the contract award to discuss details regarding the tasks outlined in the proposal, scheduling activities, and other issues related to the contract. This meeting shall be at the Department's offices in Washington, DC. The contractor shall provide an agenda for the meeting within one (1) week after award. The contractor shall come prepared to identify any areas of concern and to suggest ways of responding to these concerns. Within one (1) week following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each shall be addressed in the study.

Deliverables: Meeting Agenda Memo summarizing meeting

Task 2—Interview and OMB package Task 2.1 TSL grantees and Refine Teacher Leader Role Model

The contractor shall interview up to 9 TSL grantees that included a teacher leader role in their FY2017 grant proposal. The purpose of the interviews is to guide core aspects of the intervention to be implemented in the impact evaluation component.

Task 2.2—Prepare OMB package and Develop Data Collection Forms The contractor shall prepare all relevant OMB forms and justification for a forms clearance package that includes all data collection instruments. While the preparation of the data collection forms occurs as part of the base contract, data collection using the forms will only occur if the optional task period(s) are executed.

The package shall include a description of the study design, site recruitment, and data collection instruments, as well as the necessary forms required for OMB approval. The contractor shall ensure that the clearance package justifies the data collection activities and comprehensively responds to each required item in the OMB instructions.

In addition, the contractor shall ensure that the forms clearance package includes: (1) the study mandate and objectives, (2) the research questions, (3) description of and justification for incentives, (4) description of sampling and recruiting procedures, and (5) limitations of the study and its potential use for policy decisions.

The contractor shall submit a draft forms clearance package to the COR no later than June 2019. The COR will submit the accepted forms clearance package for clearance. Based on comments from internal and external clearance (after the first 60 day clearance process) the contractor shall make all needed revisions and submit a revised forms OMB clearance package for submission to OMB. The COR will then submit the accepted revised forms clearance package to OMB.

Data collection for the study shall consist of:

· Teacher Leader enrollment forms – The contractor shall construct forms that include background information about the teacher leaders and their prior experiences with coaching/mentoring within the impact evaluation component sample.

· Teacher Leader activity forms – The contractor shall construct forms that provide information on the extent to which teacher leaders meet with teachers and a checklist of core supports provided within the impact evaluation component sample.

· Teacher survey – The contractor shall construct a survey that provides data on teacher characteristics, the support activities that they receive, and their satisfaction with their level of support and the job more generally (the survey shall include teachers in the grades targeted by the studied strategy within the impact evaluation schools and a random sample of teachers in TSL grant schools). This survey shall be administered in the spring of the intervention year within the impact evaluation component sample. The contractor also shall administer the survey to a sample of FY2017 TSL grantee funded schools sufficient to estimate the average teacher experience among the TSL grantees.

· Student administrative records data – within the impact evaluation component sample, the contractor shall collect administrative data on student characteristics and test scores, as well as other relevant covariate and subgroup information. These data shall be collected for the year prior to the intervention implementation year (2019-20) and the year of implementation (2020-21). For TSL schools, the contractor shall use common core data and school achievement data.

· Principal survey – The contractor shall construct a survey that provides data on principal characteristics, their satisfaction with teacher supports, and the extent to which their use of teacher leader roles helps with recruiting and retaining [high-quality] teachers, especially in high-need subject areas. This survey shall be administered in the spring of the intervention year within the impact evaluation component sample. The contractor also shall administer the survey to a sample of FY2017 TSL grantee funded schools sufficient to estimate the average principal experience among the TSL grantees.

· Grantee survey – The contractor shall construct a survey that provides information about the strategies funded within the FY2017 TSL grants and provides more in-depth information about teacher leader role activities and challenges for those grantees implementing the strategy. The grantee survey shall also collect information to construct the program’s GPRA performance indicators. The survey shall be administered in spring 2020 and the spring of 2021.

The contractor shall design data collection instruments to address the key study questions. The contractor shall make use of or refine existing instruments to the extent possible and applicable.

For instruments where validity or reliability are not yet well established, the contractor shall pilot test each of the instruments with no more than nine respondents per instrument in no less than two schools to ensure that the questions can be answered in a reasonable amount of time, that the directions are easily followed, and that the contractor’s procedures and testing systems for administering the instruments are effective. Piloting is to be conducted prior to the final 60 days in the OMB submission process. The contractor shall ensure that instruments are revised as needed based on pilot test results.

The contractor shall coordinate its own Institutional Review Board (IRB) approval for the piloting, field testing, and full scale data collection with the IRB approval obtained by districts from their own institutions. Specifically the contractor is responsible for ensuring that all aspects of the program have been approved by the appropriate IRBs, including its own IRB.

Deliverables: Interview Protocol Original submission OMB clearance package (first 60 days) Revised OMB clearance package (second 60 days) Final OMB clearance package (cleared package) Documentation of IRB approvals

Task 3 - Selection and Recruitment of Study Districts and Teachers

During the selection and recruitment process, the contractor shall provide monthly selection and recruitment updates in the monthly progress reports (see Task 6).

The contractor shall, in consultation with the COR, develop a strategy for selecting study sites and prioritizing potential sites based on criteria that the COR and contractor agree are pertinent to addressing the evaluation objectives. These shall include policy-relevant districts with capabilities sufficient to support the evaluation’s needs and a willingness to support and participate in the study.

Based on the exploratory contacts with potential sites and sample selection criteria developed in consultation with the COR, the contractor shall recruit study districts and teachers to participate in and cooperate with the study. The contractor shall secure final agreements to participate through signed memoranda of understanding so that a total of approximately 10 districts, 100 schools, and up to 100 teacher leaders are included in the study sample.

The contractor shall complete all district recruitment activities no later than March 2020. Random assignment of schools to the implement the intervention and selection of teacher leaders shall occur prior to the summer teacher leader orientation session in 2020.

As part of the recruitment process, the contractor shall, as appropriate, comply with: The Privacy Act of 1974, P.L. 93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and Privacy Act of 1974, 20 USC 1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41 CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the Federal common rule or ED’s final regulations on the protection of human research participants. The contractor shall also satisfy any state, district, or school policies regarding school participation in external research projects.

Deliverables: Memoranda of Understanding from districts.

Task 4—Technical Working Group

The role of the TWG will be to advise the contractor and the Department on the conduct of the study including, but not limited to: study design, instrumentation, data collection and analysis, and draft study reports.

The TWG shall be comprised of approximately 10 individuals, and include experts in teacher support and professional development, teacher quality, teacher policy, as well as experts in evaluation methodology. The contractor shall submit a list of proposed TWG members no later than 2 months after contract award. When preparing the list, the contractor shall evaluate the potential for conflicts of interest by assessing whether each potential TWG nominee has a bias or financial interest in the outcome of the study. After the COR approves the proposed TWG members, the contractor shall contact each proposed member to secure interest and intention to serve on the TWG.

Convene Technical Working Group. The contractor shall convene a meeting of the TWG in the base contract. The TWG meeting shall be held no later than 7 months after the contract award and shall focus on remaining design issues, data collection, and analysis issues. The contractor shall prepare a study design memo that refines the study design and incorporates a detailed description of the selected treatment to be studied. The study design document will be used as materials for informing the TWG of the study design prior to the TWG meeting. A draft of the study design document shall be provided to the COR four (4) weeks prior to the TWG meeting.

After the meeting, the contractor shall submit a written summary of the meeting to the COR within one week, detailing suggestions and recommendations along with the contractor’s proposed actions. Proposed actions cannot go forward unless approved by the COR. The meeting site shall be arranged to minimize travel of Federal government staff members, with preference given to meeting in U. S. Department of Education facilities in Washington, DC. The contractor shall pay the travel, per diem, and honorarium expenses of the TWG members.

If TWG members recommend any significant deviation from the proposal, study design, instrumentation, data collection, or data analysis, the contractor shall submit the recommendation to the COR for approval prior to implementation. The contractor shall not make any significant changes in the study based solely upon the recommendation of TWG members without the written approval of the COR. The contractor also shall use the TWG members to review products, as appropriate, when group meetings are not required.

Summary of TWG meeting

Task 5.1— Implement and Monitor the Intervention Implementation

As part of the monthly reporting requirement (task 6), the contractor shall provide ongoing information about the intervention activities and as well as participant attendance information.

Fall implementation and monitoring. The contractor shall implement the teacher leader role intervention in the schools assigned to treatment. The contractor shall work with the study districts/schools to:

· Arrange a half-day orientation for recruited study teacher leaders during the summer prior to the 2020-2021 school year.

· Monitor implementation of the intervention in the fall of 2020.

Task 6- Reporting Requirements

In addition to the deliverables that are required for each task described above, the contractor shall submit one copy of the following performance reports each month to the CO and one copy to the COR, simultaneously with the monthly invoice. The base period includes performance reports for months 1 through 27.

Monthly Progress Report/Exception Reports. The contractor shall prepare monthly progress reports due at the same time as the monthly invoice. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, changes in personnel and their security designation, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from the Department. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

Monthly Manpower/Expenditure Reports. The contractor shall prepare monthly expenditure reports due at the same time as the monthly invoice. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Deliverables: Monthly progress & manpower/expenditure report

OPTIONAL TASK PERIOD I

Task 5.2— Implement and Monitor the Intervention Implementation

As part of the monthly reporting requirement (task 8), the contractor shall provide ongoing information about the intervention activities.

Late fall-spring implementation and monitoring. The contractor shall continue implementing and monitoring the teacher leader role intervention during 2020-21 in the schools assigned to treatment.

Task 7.1—Data Collection

Using instruments approved by OMB, the contractor shall collect the spring 2020 grantee survey, the teacher leader enrollment forms, and the teacher leader activity forms as described in Task 2. The contractor shall utilize appropriate techniques to obtain response rates of at least 85 percent according to the National Center for Education Statistical Standards (http://nces.ed.gov/pubs92/92021.pdf). As part of the monthly reporting requirement (task 8), the contractor shall provide ongoing information about the data collection activities.

To ensure that data collected are of high quality, the contractor shall quickly identify patterns of inaccuracies and anomalies in the data (such as out-of-range variables, missing data, and unusual distributions of scores), such that immediate corrective action can be initiated. All such problems and subsequent corrections must be carefully documented and compiled into a technical report that shall be included with the final archived data set. The data monitoring activities shall be in place within 22 months after contract award. During data collection, the contractor shall include with the monthly progress reports updates on the progress of data collection based upon the data monitoring activities.

To ensure that the data collection and coding staff have the skills necessary to perform their duties, and to ensure consistency of training across time, the contractor shall design a comprehensive training program.

The training shall document the minimum performance standards that the data collectors or coders must meet, the criteria by which the contractor shall ascertain whether data collectors and coders meet the performance standards, how often performance will be evaluated, and how the contractor will verify that the training of data collectors and coders will be consistent across all waves of data collection. All persons who collect, code, or otherwise have access to raw data must be trained in confidentiality issues and shall provide signed assurances of nondisclosure of confidential information. The contractor shall maintain these records and provide information on these procedures to the Government upon request. When possible, individuals collecting and/or coding data shall have no knowledge of each school’s treatment condition.

All persons who collect, code, or otherwise have access to raw data must be trained in confidentiality issues and shall provide signed assurances of nondisclosure of confidential information. The contractor shall maintain these records and provide them to the Government upon request.

When possible, the contractor shall ensure that individuals collecting and/or coding data do not have knowledge of the study participant’s treatment condition. The contractor, and all subcontractors, shall comply with: The Privacy Act of 1974, P.L. 93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and Privacy Act of 1974, 20 USC 1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41 CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the Federal common rule or the Department’s final regulations on the protection of human research participants. The contractor shall be responsible for safeguarding any PII that is collected for the purposes of this contract. The contractor must also comply with any relevant requirements of the Privacy Act, 5 U.S.C. 552a; the E-Government Act of 2002, 44 U.S.C. §208; as well as relevant OMB directives, such as OMB M-06-16 and OMB M-17-12. The Contractor shall abide by and follow all Departmental privacy policies, procedures, processes, and standards. All electronically stored sensitive data shall be encrypted at rest and in motion. The contractor shall also satisfy any state, district, or school policies regarding school participation in external research projects. Contractor and subcontractor employees who “maintain” (collect, maintain, use, or disseminate) data shall comply with the requirements of the confidentiality standards in section 183 of the Education Sciences Reform Act of 2002 (ESRA) (20 U.S.C. 9573).

Task 8- Reporting Requirements In addition to the deliverables that are required for each task described above, the contractor shall submit one copy of the following performance reports each month to the CO and one copy to the COR, simultaneously with the monthly invoice. The optional task period I phase includes performance reports for months 28 through 40.

Monthly Progress Report/Exception Reports. The contractor shall prepare monthly progress reports due at the same time as the monthly invoice. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, changes in personnel and their security designation, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

Monthly Manpower/Expenditure Reports. The contractor shall prepare monthly expenditure reports due at the same time as the monthly invoice. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

OPTIONAL TASK PERIOD II

Task 7.2 — Data Collection (con’t)

Using instruments approved by OMB, the contractor shall collect all remaining study administered data (teacher survey, principal survey, spring 2021 grantee survey, administrative records) to estimate the impact of the intervention on the outcomes of interest and to answer the evaluation questions. The contractor shall utilize appropriate techniques to obtain response rates of at least 85 percent according to the National Center for Education Statistical Standards (http://nces.ed.gov/pubs92/92021.pdf).

To ensure that data collected are of high quality, the contractor shall use all procedures outlined in task 7.1 above and provide monthly information about data collection activities as part of the monthly report in task 11.

Task 9.1—Data Analyses, TSL Implementation and GPRA memo I

The contractor shall conduct data analyses and begin preparing a draft report that clearly addresses the evaluation questions.

The contractor shall make sure that the evaluation report is written and organized, so that it is accessible to policy makers and research-savvy practitioners rather than academic researchers. Publications include

· A one-pager. This shall be a stand-alone introduction to the report and contain enough information to form the basis for a press release.

· A four-pager. This document shall convey the key findings and features of the report suitable for ED program staff, reporters, and policymakers (Congress and Hill staffers). It shall include key graphics and highlights.

· A 15-page report. Full technical aspects of the report shall be included in appendices.

The contractor shall prepare an evaluation brief of the TSL program implementation and a memo that provides information about the GPRA indicators 2 months after the completion of the spring 2020 grantee survey data collection.

Deliverables: Implementation evaluation brief and GPRA memo I (based on spring 2020 data)

Task 10—Technical Working Group

Convene Technical Working Group. The contractor shall convene a meeting of the TWG members to discuss the study findings. The meeting shall be held no later than March 2022. After the meeting, the contractor shall submit a written summary of the meeting to the COR within one week, detailing suggestions and recommendations along with the contractor’s proposed actions. Proposed actions cannot go forward unless approved by the COR. The contractor shall pay the travel, per diem, and honorarium expenses of the TWG members, and any lodging required by the TWG members. Finally, the COR will secure a meeting space at Department facilities.

The contractor shall provide copies of all reviews of products by the panel to the COR within five (5) work days of receipt by the contractor. If TWG members recommend any significant deviation from the data analysis plan, the contractor shall submit the recommendation to the COR for approval prior to implementation. The contractor shall not make any significant changes in the study based solely upon the recommendation of TWG members without the written approval of the COR.

Deliverables: Summary of TWG meeting

Task 11—Reporting Requirements

In addition to the deliverables that are required for each task described above, the contractor shall submit one copy of the following performance reports each month to the CO and one copy to the COR, simultaneously with the monthly invoice. The optional task period II phase includes performance reports for months 41 through 44.

Monthly Progress Report/Exception Reports. The contractor shall prepare monthly progress reports due at the same time as the monthly invoice. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, changes in personnel and their security designation, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

Monthly Manpower/Expenditure Reports. The contractor shall prepare monthly expenditure reports due at the same time as the monthly invoice. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Deliverables: Monthly progress & manpower/expenditure report

OPTIONAL TASK PERIOD III

Task 9.2—Report and Data File

The contractor shall submit a draft report no later than 4 months after the TWG meeting (Task 10). Within three weeks, the COR will review and provide comments on the draft report. The contractor shall submit a revised draft report ready for submission to the Standard’s and Review Office (SRO) within 2 months of the Department comments. The Department will submit the draft report to the SRO. Within approximately 2 months, the Department will provide comments to the contractor from the SRO.

The contractor shall make final revisions based on the SRO comments and submit a revised final first report within one month. The contractor shall revise the report until all peer review comments are addressed and the report clears the SRO review (current experience is an average of 3 rounds of successively less extensive comments). The contractor shall provide a web-ready and a print ready version of the report to the COR within 2 weeks after the report is cleared by the SRO.

Deliverables: Draft report Revised draft report, SRO-ready

Task 12 – Grantee GPRA memo II

The contractor shall prepare a memo that provides information for the TSL program GPRA indicators 2 months after the completion of the second round of grantee survey data collection (Task 9).

Deliverables: GPRA memo II (based on spring 2021 data)

Task 13—Reporting Requirements

In addition to the deliverables that are required for each task described above, the contractor shall submit one copy of the following performance reports each month to the CO and one copy to the COR, simultaneously with the monthly invoice. The optional task period III phase includes performance reports for months 45 through 59.

Monthly Progress Report/Exception Reports. The contractor shall prepare monthly progress reports due at the same time as the monthly invoice. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

Monthly Manpower/Expenditure Reports. The contractor shall prepare monthly expenditure reports due at the same time as the monthly invoice. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Deliverables: Monthly progress & manpower/expenditure report

INFORMATION TECHNOLOGY AND INFORMATION SECURITY REQUIREMENTS

The contractor shall ensure that all data in a computerized form shall be suitable for data editing and corrective actions consistent with the U.S. Department of Education’s Information Quality Guidelines (http://www.ed.gov/offices/OCIO/info_quality/final_guide.html) in order to provide for quality control of data entry and yielding verified and accurate records.

The contractor shall ensure that all reports—including the final reports prepared under task 9—comply with the appropriate technical standards for compliance with Section 508 of the Rehabilitation Act of 1973 as amended, at 36 CFR 1994, and be subject to conformance testing by the Department’s Assistive Technology Program. All Section 508 violations uncovered during the testing process shall be remediated by the vendor at no charge to the Department.

The Department has determined that the work under this contract does not involve a federal information system and therefore does not require a Security Authorization (SA) or Authority to Operate (ATO) designation by the Department. However, with regard to information obtained in carrying out the work requirements, the contractor, and all subcontractors, shall maintain a level of security that is similar to what is specified by the Federal Information Security Modernization Act (FISMA) of 2014, Office of Management and Budget (OMB) Circular A-130, Managing Information as a Strategic Resource, the most current, finalized version of the National Institute of Standards and Technology (NIST) standards and guidance SP 800-53 and SP 800-53A, and the Federal Risk and Authorization Management Program (FedRAMP) requirements and guidance. Guidance on key components of information security can be found here: https://www.nist.gov/programs-projects/federal-information-security-management-act-fisma-implementation-project

Specifically, all contractors and subcontractors shall ensure that:

1. Their IT product/system is monitored during all hours of operations using entrusted detective/preventive systems;

1. Their IT product/system has current antiviral products installed and operational;

1. Their IT product/system is scanned on a reoccurring basis;

1. Vulnerabilities are remediated in a timely manner on their IT product/system; and

1. Access/view for cyber security situational awareness on their IT product/system is made available to the Department CIRC (cyber incident response capability).

Preparing for and Responding to a Breach of Personally Identifiable Information Per OMB-M-17-12:

• The contractor shall cooperate with and exchange information with agency officials, as determined necessary by the agency, in order to effectively report and manage a suspected or confirmed breach.

• The contractor and subcontractors (at any tier) shall properly encrypt PII in accordance with OMB Circular A-130 and other applicable policies and to comply with any agency-specific policies for protecting PII;

• The contractor shall complete regular Department training for contractors and subcontractors (at any tier) on how to identify and report a breach;

• The contractor and subcontractors (at any tier) shall report a suspected or confirmed breach in any medium or form, including paper, oral, and electronic, as soon as possible and without unreasonable delay, consistent with the agency's incident management policy and US-CERT notification guidelines;

• The contractor and subcontractors (at any tier) shall maintain capabilities to determine what Federal information was or could have been accessed and by whom, construct a timeline of user activity, determine methods and techniques used to access Federal information, and identify the initial attack vector;

• The contractor shall allow for an inspection, investigation, forensic analysis, and any other action necessary to ensure compliance with this Federal and Department PII Breach Response policies (such as OMB-M-17-12), the Department’s breach response plan, and to assist with responding to a breach;

• The contractor shall identify roles and responsibilities, in accordance with Federal and Department PII Breach Response policies (such as OMB-M-17-12), and the agency's breach response plan; and,

• The contractor shall be aware that a report of a breach shall not, by itself, be interpreted as evidence that the contractor or its subcontractor (at any tier) failed to provide adequate safeguards for PII.

Reporting of data security breaches

If there is a suspected or known breach/disclosure of personally identifiable information (PII) related to this contract due to lost, theft, intercepted transfer, or other reason, the contractor must ensure that this breach is reported to the agency as soon as the contractor has knowledge of it. Per Office of Management and Budget Memorandum M-17-12, Federal agencies have a requirement to report breaches of PII security to a United States Computer Emergency Response Team (US-CERT). IES must notify the department within 30 minutes of discovering the incident (and the agency should not distinguish between suspected or confirmed breaches). The data security plan must be written to reflect this requirement, and the contractor must provide sufficient notification and documentation of the suspected loss, as it is understood at the time of notification to the agency for this requirement to be met. Follow-up reports of the final status of loss events will also be prepared by the contractor within a reasonable period of time as advised by the COR.

SCHEDULE OF DELIVERABLES

All deliverables under these tasks shall be submitted to the Department’s Contracting Officer’s Representative (COR), the Contracting Specialist (CS), and the Contracting Officer (CO) electronically unless otherwise noted. Unless indicated otherwise, the COR will notify the contractor within two weeks (unless otherwise noted in the Schedule of Deliverables) of receipt of any changes required in the deliverables, and the contractor shall have up to two weeks to make the necessary revisions and submit a final deliverable to the Department.

Deliverable Schedule

Date

Base Period

Task 1 – Opening Meeting with the Department

Task 2 – Prepare OMB package and Develop Data Collection Forms

Task 3 – Selection and Recruitment of Study Districts, Schools, and Teacher Leaders

Task 4 – Technical Working Group

Task 5.1 – Implement and Monitor the Intervention Implementation

Task 6 – Reporting Requirements

Optional Task Period I

Task 5.2 – Implement and Monitor the Intervention Implementation

Task 7.1 – Data Collection

Task 8 – Reporting Requirements

Optional Task Period II

Task 7.2 – Data Collection

Task 9.1 – Analyses, GPRA Memo,…