Title I II SOW 060811.doc

Draft Performance-based Work Statement

Implementation of Title I/II Program Initiatives National Center for Education Evaluation

Institute of Education Sciences

U.S. Department of Education

I. INTRODUCTION

The Institute of Education Sciences (IES) at the U.S. Department of Education (ED) is conducting a broad assessment of the implementation of Title I and Title II programs under the Elementary and Secondary Education Act (ESEA). As part of this effort, IES intends to award a 5-year contract with a 2-year option to conduct an implementation evaluation of how states, schools, and districts are using Title I and Title II funds. We also intend to look at student outcome trends pre- and post-reauthorization of ESEA. The program office responsible will be the National Center for Education Evaluation (NCEE) of the Institute of Education Sciences. This Performance Work Statement includes a base contract with a period of performance of 60 months and an option for an additional round of follow-up data collection which would extend the contract to 84 months.

A. Legislative Authority Title I, Part E, Section 1501 of the No Child Left Behind Act has mandated a National Assessment of the implementation and impact of Title I in the past, and this Title I study is planned in anticipation of the next reauthorization of ESEA. The Title II study is authorized in Title II, Part A of the Elementary and Secondary Education Act (ESEA, section 2121-2123 as amended by No Child Left Behind (NCLB) (20 USC 6621-6623).

B. Background Title I and Title II were created as part of ESEA, which was originally enacted in 1965 to help provide all students with equal access to education. It was intended to ensure that all students meet certain standards, and in particular, assist those students with the lowest levels of achievement. The No Child Left Behind (NCLB) Act of 2001 expanded upon ESEA by increasing accountability through the use of assessments for students, requirements for all teachers to be highly qualified, and funding for supports and interventions for schools not achieving adequate yearly progress for the entire school population as well as for various subgroups of students.

Currently, more than 50,000 public schools use Title I funds to help low-achieving students increase achievement and meet state standards.

Historically, there has been a Congressionally-mandated study of Title I which has also included information about Title II; this study is planned in anticipation of the upcoming reauthorization of ESEA. While the timeline for reauthorization is uncertain, reauthorization is overdue and could happen as early as within the next year. Based on the likely areas of concentration as signaled by the administration’s publication “A Blueprint for Reform” (http://www2.ed.gov/policy/elsec/leg/blueprint/blueprint.pdf) and ongoing reauthorization discussions, this study is designed to provide relevant baseline data prior to the reauthorization as well as follow-up data to evaluate the implementation of a reauthorized Title I and Title II.

C. Programs This evaluation will focus on implementation of Title I and Title II.

Title I

Title I has historically provided financial assistance to schools and districts which have a high percentage of students from low-income families. It is designed to help these students increase achievement and reach proficiency on state academic achievement standards, and hold schools and districts accountable for improvements in student academic achievement. The administration’s proposal for reauthorization of ESEA has focused on: developing standards and assessments that are focused on college- and career-readiness, providing equitable access to high-quality education for all students, how states are identifying and rewarding the highest-performing schools, and the types of assistance and/or turnaround models being implemented in the lowest-performing schools.

Title II

The purpose of the Title II program is to increase academic achievement by improving teacher and principal quality. This program has historically focused on: providing students with a “highly qualified” teacher in every classroom and increasing the effectiveness of teachers and principals by holding districts and schools accountable for improvements in student academic achievement. The requirements for a “highly qualified” teacher currently include a bachelor’s degree, state certification, and demonstration of content knowledge. Prior uses of Title II have focused on improvements in teacher quality primarily through supporting funding for professional development and class size reductions. Other allowable uses of Title II funding in the current authorization include support for teacher and principal recruitment and hiring, induction, and retention.

Going forward, the administration has proposed consolidating funding across a number of teacher programs (including Title II) in the next reauthorization of ESEA with a focus on giving states more flexibility in meeting the goal of improving student outcomes through supporting and rewarding effective teachers and leaders. States would create their own definitions of “effective” and “highly effective” teachers and leaders, based on outcome measures such as student achievement growth and classroom observations, and take steps to ensure that students have equitable access to such teachers and leaders. States would also implement evaluation systems to measure teacher and leader quality and meaningfully differentiate teachers and principals by effectiveness.

Related Programs and Studies Although there is no current IES study specifically focusing on Title I and Title II implementation, there are a number of current IES studies that will provide related information that focus on ED’s reform agenda, such as ED’s implementation study of the American Recovery and Reinvestment Act (ARRA). The expected key areas of interest in Title I and Title II are consistent with ARRA’s four core reform areas: (1) adopting rigorous college- and career-ready standards and high quality assessments, (2) establishing data systems and using data to improve performance, (3) increasing teacher effectiveness and the equitable distribution of effective teachers, and (4) turning around low-performing schools.

ED is also conducting evaluations of three components of ARRA: Race to the Top (RTT), School Improvement Grants (SIG), and the Teacher Incentive Fund (TIF). RTT is a competitive grant program designed to encourage states that are creating the conditions for education innovation and reform, improving student outcomes, and implementing comprehensive plans in the four core reform areas of ARRA. SIG addresses ARRA’s fourth core reform area and is intended to improve student achievement in the lowest-achieving schools through the adoption of 1 of 4 school turnaround models. ED’s combined SIG/RTT evaluation focuses on implementation, the impacts of SIG funding on student outcomes, and the relationships of RTT grants with student outcomes. The TIF evaluation addresses ARRA’s third core reform area by measuring the impacts of the program’s performance-based teacher and principal compensation systems, based in part on growth in student achievement, on outcomes in high-need schools.

The Title I and Title II implementation study will delve deeper into the issues related to the reauthorization of ESEA than these related studies and will collect data for a representative sample of districts, schools, and teachers over a longer period of time than some of these studies. The contractor should be aware of the data collection timelines for these related studies in order to coordinate any similar data collection and minimize overlapping samples. The ARRA evaluation will be surveying districts and states in spring 2011, spring 2012, spring 2013, and with an option of spring 2014. The RTT/SIG evaluation will be collecting data at the state, district, and school levels in 2011-2012, 2012-2013, and 2013-2014. Data collection for the TIF evaluation will take place in 2011-2012, 2012-2013, 2013-2014, and 2014-2015. The sample for the Title I and Title II study will overlap in states and possibly in districts with earlier studies. The contractor shall leverage any relevant information from the related studies and limit the data collection burden in states and in any overlapping schools and districts.

D. Key Research Questions The main purpose of this evaluation is to describe progress on the core teacher, leader, and school improvement policies promoted through ESEA and to determine how states, districts, and schools are using Title I and Title II funds under the reauthorization of ESEA. Based on current conversations regarding key areas to be included in the reauthorization, information gathered shall address the following topic areas:

· College- and Career-Ready Students. How are states, districts, and schools adopting standards and aligning assessments that reflect the goal of all students graduating from high school college- and career-ready, and to what extent are they implementing them? How are student resources and outcomes distributed within each state?

· Teachers and Leaders. How are states, districts, and schools defining and evaluating “effective” and “highly effective” teachers and principals? Do all students have equitable access to effective teachers and principals, and what are states, districts, and schools doing to address this goal? How are states strengthening the pathways to teaching, preparing and supporting teachers and principals, and retaining effective teachers and principals?

· Progress on Growth Toward Raising Achievement. What accountability systems have states implemented? How are students making progress on meeting state academic achievement standards within states, and how are students performing across states? How are states identifying and rewarding the highest-achieving schools?

· Turning Around Low-Performing Schools. How are states identifying the lowest-performing schools? What types of schools are not meeting standards, and why? What types of assistance and interventions are states and districts using in the lowest-performing schools? What turnaround models and strategies are being implemented in schools?

The topic areas of interest in this evaluation may shift as reauthorization negotiations evolve. The contractor shall adjust the key research questions accordingly as these reauthorization discussions unfold.

E. Study Approach The primary purpose of this study is to describe progress on the core teacher, leader, and school improvement policies promoted through ESEA and to determine how states, districts, and schools are using Title I and Title II funds under the reauthorization of ESEA. The study is primarily descriptive and will use surveys along with existing data. When conducting the evaluation to address the key research questions, the contractor shall consider the following assumptions:

· The contractor shall survey all states regarding the implementation of their accountability and assessment systems as well as perform document reviews of materials related to states’ policies and guidelines regarding measures, progress, and accountability for student achievement and teacher effectiveness.

· The contractor shall conduct 2 waves of surveys – one baseline and one follow-up – of a nationally representative sample of districts and schools. The initial baseline survey shall be conducted in the 2012-2013 school year, and the follow-up survey shall be conducted 2 years following the effective date of the reauthorization of ESEA (approximately the 2014-2015 school year). There is also an option for a 2nd follow-up survey in Option #1, which would be conducted 4 years following the effective date of the reauthorization (approximately the 2016-2017 school year). ED will decide within 48 months of the contract award if Option #1 will be exercised. Based on preliminary calculations, ED expects a nationally representative sample of approximately 300 districts and 1,500 schools. The contractor shall also survey 1 principal and approximately 6-7 teachers within these schools.

· In order to minimize the burden on state, district, and school personnel, the contractor shall also use other relevant sources of data in addition to surveys. The contractor shall also consider using one or more datasets that provide consistent measures of student achievement across states. Because each state has developed its own standards and assessments, comparing student achievement across states using state assessment data is not possible. One such dataset that would allow the comparison of student achievement across states – and nationally – is the National Assessment of Educational Progress (NAEP), which provides data on national trends in student achievement.

F. Study Issues

A number of potential challenges may affect the study design and implementation. The contractor shall consider these issues while carrying out the study:

· Reauthorization Uncertainty. The timing of, and language included in, the reauthorization of ESEA is uncertain. The contractor may need to adapt the timing and the focus of follow-up data collection based on any unforeseen changes. The contractor shall consider this uncertainty when developing the baseline data collection and outline an ability to be flexible with staffing.

· Sampling. The contractor shall develop a strategy for sampling districts, schools, teachers, and principals to best address the key research questions.

· Data Collection. The contractor shall consider how to limit the burden imposed on states, districts, and schools despite a desire for detailed information from multiple stakeholders. The contractor shall also consider how to ensure that responses to survey items are reliable and valid.

· Coordinating with Existing Studies. Existing studies, such as the ARRA evaluation and the RTT/SIG evaluation, will be conducting state surveys and may be conducting district surveys in some of the same districts as this study. The contractor shall leverage any relevant information from these studies and limit the data collection burden in states and in any overlapping districts.

II. SCOPE OF WORK

The Contracting Officer’s Representative (COR) reserves the right to inspect the contractor’s work while in process. The contractor shall submit to the COR any progress report or other written documentation of work to date that the COR requests to facilitate that inspection, according to contract clause 52.246-4 (fixed price) and 52.246-5 (cost-reimbursement).

This section specifies the tasks and subtasks that the contractor shall perform. All deliverables under these tasks shall be submitted to ED’s Contracting Officer’s Representative (COR), ED’s Contracting Specialist (CS), and ED’s Contracting Officer (CO) electronically unless otherwise noted. Unless indicated otherwise, the COR will notify the contractor within two weeks of receipt of any changes required in the deliverable, and the contractor shall have up to two weeks to make the necessary revisions and submit a final deliverable to ED unless otherwise noted.

A. Design Phase

Task 1—Meeting with ED (Firm Fixed Price Task)

1.1 Opening Meeting. The contractor’s project director and any key project staff shall meet with the Department of Education’s (ED) Contracting Officer’s Representative (COR) and other appropriate ED staff within 15 work days after the effective date of contract award to discuss details regarding the tasks outlined in the proposal, study design, scheduling activities, and other issues related to the contract. This meeting shall be at ED’s offices in Washington, DC. The contractor shall come prepared to identify any areas of concern and to suggest ways of responding to these concerns. Within 1 week following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting and how each will be addressed in the study.

Deliverable: Memo summarizing meeting

Task 2—Technical Working Group for the Evaluation (Firm Fixed Price Task)

2.1 Establish Technical Working Group (TWG). The role of the TWG will be to advise the contractor and IES on the conduct of the study including, but not limited to: evaluation design, instrumentation, data collection and analysis, and presentation of evaluation findings. The TWG shall be comprised of up to 10 individuals. The contractor shall evaluate the potential for conflicts of interest by assessing whether a potential TWG member has a bias or financial interest in the outcome of the study. The contractor shall discuss proposed panelists with IES at the opening meeting. After IES endorses the proposed TWG members, the contractor shall contact each proposed member and obtain agreement to participate on the TWG by the third month after the effective date of the contract.

2.2 Convene Technical Working Group. The contractor shall convene up to 3 meetings of the TWG over the period of the contract. The first meeting shall be held no later than 6 months after the contract award and shall focus on the proposed design for the evaluation. The contractor, in conjunction with ED, shall decide upon the timing of the second and third meetings, which shall be held after preliminary analyses of data from the baseline and follow-up data collections have been conducted. No later than 2 weeks before each meeting, the contractor shall submit to the COR draft materials for the meeting, including an agenda, presentation materials, and other relevant materials for the panel’s review. The COR will review and comment within 1 week. Final materials shall be fully responsive to COR comments on the draft materials. The contractor shall send final materials to the TWG members no later than 1 week before each meeting.

The contractor shall submit a written summary of each meeting to the COR within 2 weeks following each meeting, detailing suggestions and recommendations along with proposed actions on those suggestions. Meeting sites shall be arranged to minimize travel of Federal government staff members. The contractor shall cover the travel, per diem and honorarium expenses of the TWG members.

The contractor shall provide copies of all reviews of products by the panel to the COR within five work days of receipt by the contractor. If TWG members recommend any significant deviation from the proposal, study design, instrumentation, data collection, or data analysis, the contractor shall submit the recommendation to the COR for approval prior to implementation. The contractor shall not make any significant changes in the study design based solely upon the recommendation of a TWG member without the written approval of the COR. The contractor also shall use the TWG members to review products, as appropriate, when group meetings are not required.

Deliverables: Summaries of advisory panel meetings

Task 3—Develop Sampling Plan (Firm Fixed Price Task) The contractor shall develop a strategy for obtaining a nationally representative sample of districts and schools from the universe of public school districts in the United States for the COR’s review and approval. The contractor shall provide a sample design and sample size that maximizes the generalizability of the data and allows for disaggregation by subgroups of interest. Based on preliminary calculations conducted by ED, it is anticipated that the national sample will include approximately 300 school districts and 1,500 schools in the United States. The sampling plan shall minimize overlap with ARRA and RTT/SIG samples, to the extent feasible, and provide a justification for overlaps.

The contractor shall submit a sampling plan in memo format to the COR within 14 weeks of the contract award. The COR will review and provide comments within 1 week. The contractor shall revise the sampling plan memo to reflect the COR’s comments for final acceptance. This sampling plan shall be incorporated into the baseline forms clearance package in Task 5.

Deliverable:

Draft memo describing sampling plan

Final memo describing sampling plan

Task 4—Revise Study Design (Firm Fixed Price Task) The contractor shall revise the study design submitted in the proposal to include the policy relevance of such a study, evaluation methodology, data collection, and data analysis plans for the evaluation based on TWG feedback. The revised study design shall be completed within 1 month after the initial TWG meeting.

Deliverable: Revised study design Task 5—Develop Data Collection Forms and Prepare OMB Package

5.1 Development of Data Collection Forms. (Cost Reimbursement Subtask) The contractor shall collect data through surveys, document reviews, and existing data. The contractor shall design data collection forms to address the key research questions.

Data collection shall consist of:

· State Surveys: to gain an understanding of the standards and assessments states have implemented as well as what types of accountability systems they have in place; how states are defining “effective” and “highly effective” teachers and leaders and holding them accountable; and what types of turnaround models states are using in the lowest-performing schools and what types of rewards in the highest-achieving schools.

· District Surveys: to determine how schools and districts are encouraging all students to graduate college- and career-ready; how and why schools are or are not making progress toward that goal; the extent to which teacher and student resources and outcomes are distributed equitably; and what types of assistance and interventions are used in low-performing schools and for low-performing students.

· Teacher Surveys: to document teacher pathways; teacher preparation and support, including professional development; and what processes are implemented for retaining and rewarding “effective” and “highly effective” teachers.

· Principal Surveys: to gain an understanding of how schools are using Title I and Title II funds; how principals assess teacher quality and what teacher accountability measures schools have adopted; hiring, termination, and transfer decisions; how principals are rewarding and retaining “effective” and “highly effective” teachers; and the student and teacher composition of schools.

The contractor shall pilot test each of the instruments with no more than 9 respondents per instrument in no less than 2 schools to ensure that the questions can be answered in a reasonable amount of time, that the directions are easily followed, and that the contractor’s procedures and testing systems for administering the instruments are effective. Piloting is to be conducted prior to the final 60 days in the OMB submission process. The contractor shall ensure that instruments are revised as needed based on pilot test results.

The contractor shall coordinate its own Institutional Review Board (IRB) approval for the piloting, field testing, and full scale data collection with the IRB approval obtained by districts from their own institutions. Specifically the contractor is responsible for ensuring that all aspects of the program have been approved by the appropriate IRBs, including its own IRB.

5.2 Prepare Baseline OMB Package Materials. (Firm Fixed Price Subtask) The contractor shall prepare all relevant forms and justification for a clearance package that includes all baseline data collection instruments. The COR will review the draft data collection forms within 2 weeks. The baseline OMB package shall include a description of the study design and data collection instruments, and include the necessary forms required for OMB approval. The contractor shall ensure that the clearance package justifies the data collection activities and comprehensively responds to each required item in the OMB instructions. In addition, the contractor shall ensure that the forms clearance package includes: (1) the study mandate and objectives, (2) the research questions, (3) description of and justification for incentives, (4) description of sampling and recruiting procedures, and (5) limitations of the study and its potential use for policy decisions. The contractor shall submit a baseline OMB clearance package to the COR by March 1, 2012 and the revised baseline OMB package by July 1, 2012.

Deliverables:

Baseline OMB clearance package

Revised baseline OMB clearance package

5.3 Prepare Follow-Up OMB Package Materials. (Firm Fixed Price Subtask) The contractor shall prepare all relevant forms and justification for a clearance package that includes all follow-up data collection instruments. The contractor shall design the follow-up data collection instruments to capture any relevant changes in policies, guidelines, or practices resulting from a reauthorization of ESEA. The COR will review the draft data collection forms within 2 weeks. The follow-up OMB package shall include a description of the study design and data collection instruments, and include the necessary forms required for OMB approval. The contractor shall ensure that the clearance package justifies the data collection activities and comprehensively responds to each required item in the OMB instructions. In addition, the contractor shall ensure that the forms clearance package includes: (1) the study mandate and objectives, (2) the research questions, (3) description of and justification for incentives, (4) description of sampling and recruiting procedures, and (5) limitations of the study and its potential use for policy decisions. The contractor shall submit a follow-up OMB clearance package to the COR by March 1, 2014 and the revised follow-up OMB package by July 1, 2014.

Deliverables:

Follow-up OMB clearance package

Revised follow-up OMB clearance package B. Evaluation Phase

Task 6—Data Collection (Cost Reimbursement Task) Using instruments approved by OMB, the contractor shall collect data (listed in Task 5) to examine the implementation of Title I and Title II. The contractor shall utilize appropriate techniques to obtain response rates of at least 85 percent according to the National Center for Education Statistical Standards (http://nces.ed.gov/pubs92/92021.pdf ). The contractor shall also conduct document reviews of materials related to the documentation of states’ policies and guidelines regarding measures, progress, and accountability for student achievement and teacher effectiveness. Progress on data collection shall be documented in the monthly reports under Task 8.

The contractor, as appropriate, shall comply with: The Privacy Act of 1974, P.L. 93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and Privacy Act of 1974, 20 USC 1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41 CFR Part 1-1 and 45 CFR Part 5b.

All persons who collect, code, or otherwise have access to raw data must be trained in confidentiality issues and shall provide signed assurances of nondisclosure of confidential information. The Contractor shall maintain these records and provide them to the Government upon request.

Task 7—Data Analysis and Report Preparation (Cost Reimbursement Task) The contractor shall conduct data analysis and prepare 2 reports based on data collected as part of the Title I and Title II implementation study. The purpose of the report shall be to document the implementation of Title I and Title II funding under a reauthorized ESEA.

The first report shall include data analysis based on the baseline data collection. The contractor shall submit a draft report on study findings no later than January 15, 2014. The COR will review and provide comments within 3 weeks. The contractor shall submit a revised draft report ready for peer review by March 30, 2014. ED will submit for peer review. Within approximately 2 months, ED will provide comments to the contractor from the IES peer review. The contractor shall make final revisions based on IES peer review comments and submit a revised final baseline report within 1 month. The contractor shall revise the report until all peer review comments are addressed and the report clears peer review (current experience is an average of 3 rounds of successively less extensive peer review comments). The contractor shall provide a web-ready and a print ready version of the report to the COR within 2 weeks after the report is cleared by peer review.

The contractor shall also prepare a restricted use data file containing all data collected for the evaluation with state/district/school identifiers removed. The contractor shall provide documentation for the file including variable labels, category labels and weights, and how missing data were addressed. In addition, documentation accompanying the data shall include the unit of observation for each data file (e.g., state, district, school, etc.). The contractor shall also provide IES with a risk disclosure analysis for submission to NCES at the time of a revision to peer review. Once the codebook and risk disclosure analysis are approved, the contractor shall submit to the COR a restricted use data file consistent with NCES standards (http://nces.ed.gov/statprog/style.asp) at the same time as the web-ready and print ready version of the final baseline report. ASCII or SAS file formats are acceptable.

The second report shall include data analysis based on the follow-up data collection and analyses of NAEP. The contractor shall provide specifications for the NAEP analyses to the COR for submission to NCES to perform the analyses by January 15, 2015. The contractor shall submit a draft report on study findings no later than January 15, 2016. The COR will review and provide comments within 3 weeks. The contractor shall submit a revised draft report ready for peer review by March 30, 2016. ED will submit for peer review. Within approximately 2 months, ED will provide comments to the contractor from the IES peer review. The contractor shall make final revisions based on IES peer review comments and submit a revised final baseline report within 1 month. The contractor shall revise the report until all peer review comments are addressed and the report clears peer review (current experience is an average of 3 rounds of successively less extensive peer review comments). The contractor shall provide a web-ready and a print ready version of the report to the COR within 2 weeks after the report is cleared by peer review.

The contractor shall provide a follow-up restricted use data file, a data analysis plan, and a codebook, as specified above for the baseline report, at the same time as the web-ready and print ready version of the final baseline report.

Any and all documents produced under this work statement which are to be posted on the web shall be compliant with the Section 508 accessibility standards for individuals with disabilities posted at 36 CFR 1194.22, and shall be tested for accessibility by the Department’s Assistive Technology Program.

During the course of the study and at its completion, the contractor shall be prepared to present approximately two briefings of the results for program staff and/or policy makers prior to or in conjunction with the release of the baseline and follow-up reports.

Deliverables:

Draft baseline report

Draft final baseline report

Final baseline report

Baseline data file, data analysis plan, and codebook Draft follow-up report

Draft final follow-up report

Final follow-up report

Follow-up data file, data analysis plan, and codebook

Task 8—Reporting Requirements (Firm Fixed Price Task)

In addition to the reports that are required for each task described above, the contractor shall submit one copy of the following reports each month to the CO and one copy to the COR:

· Monthly Progress Report/Exception Reports. The contractor shall prepare monthly progress reports due within 10 work days after the end of each month. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

· Monthly Manpower/Expenditure Reports. The contractor shall prepare monthly expenditure reports due within 10 work days after the end of each month. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Deliverables: Monthly progress reports C. Option Option #1: Second Follow-Up (2016-2017) Data Collection

(If this option were to be executed, it would likely be within 48 months of contract award.)

If ED executes Option #1, the contractor shall administer a second follow-up of state, district, and school level surveys in the 2016-2017 school year.

Task 9—Communicate with ED (Firm Fixed Price Task) The contractor shall prepare monthly reports and submit the reports to the COR and the CS within 10 work days after the end of each calendar month of Option #1. Monthly Progress Report/Exception Reports and Monthly Manpower/Expenditure Reports shall be submitted as outlined above under Task 8.

Deliverables:

Monthly progress reports Task 10—Technical Working Group (Firm Fixed Price Task) The contractor shall convene 1 meeting of the TWG over the period of Option #1. The contractor, in conjunction with ED, shall decide upon the timing of the meeting, which shall be held after preliminary analyses of data from the second follow-up data collection have been conducted. No later than 2 weeks before the meeting, the contractor shall submit to the COR draft materials for the meeting, including an agenda, presentation materials, and other relevant materials for the panel’s review. The COR will review and comment within 1 week. Final materials shall be fully responsive to COR comments on the draft materials. The contractor shall send final materials to the TWG members no later than 1 week before the meeting.

The contractor shall submit a written summary of the meeting to the COR within 2 weeks following the meeting, detailing suggestions and recommendations along with proposed actions on those suggestions. Meeting sites shall be arranged to minimize travel of Federal government staff members. The contractor shall cover the travel, per diem and honorarium expenses of the TWG members.

The contractor shall provide copies of all reviews of products by the panel to the COR within five work days of receipt by the contractor. If TWG members recommend any significant deviation from the proposal, study design, instrumentation, data collection, or data analysis, the contractor shall submit the recommendation to the COR for approval prior to implementation. The contractor shall not make any significant changes in the study design based solely upon the recommendation of a TWG member without the written approval of the COR. The contractor also shall use the TWG members to review products, as appropriate, when group meetings are not required.

Deliverable: Summary of advisory panel meeting Task 11—Data Collection (Cost Reimbursement Task) The contractor shall use the same data collection plans and instruments as specified in the approved follow-up forms clearance package under Task 5. The state surveys shall be administered by the contractor to all states. The district and school level surveys will be administered to the nationally representative sample of high-poverty districts and Title I schools within these districts. As described earlier, ED’s preliminary calculations indicate that approximately 300 districts and 1,500 schools from the fifty U.S. states should be included in the representative sample. Progress on data collection shall be documented in the monthly reports under Task 9.

The contractor shall follow the data collection guidelines outlined above under Task 6.

Task 12—Data Analysis and Report Preparation (Cost Reimbursement Task) The contractor shall analyze the data from the surveys in the same manner as approved in the analysis plan under Task 7.

The contractor shall conduct data analysis and prepare 1 report based on data collected as part of Option #1. The report shall include data analysis based on the second follow-up data collection and analyses of NAEP. The contractor shall provide specifications for the NAEP analyses to the COR for submission to NCES to perform the analyses by January 15, 2017. The contractor shall submit a draft second follow-up report on study findings no later than January 15, 2018. The COR will review and provide comments within 3 weeks. The contractor shall submit a revised draft report ready for peer review by March 30, 2018. ED will submit for peer review. Within approximately 2 months, ED will provide comments to the contractor from the IES peer review. The contractor shall make final revisions based on IES peer review comments and submit a revised second follow-up report within 1 month. The contractor shall revise the report until all peer review comments are addressed and the report clears peer review (current experience is an average of 3 rounds of successively less extensive peer review comments). The contractor shall provide a web-ready and a print ready version of the report to the COR within 2 weeks after the report is cleared by peer review.

The contractor shall provide a follow-up restricted use data file, a data analysis plan, and a codebook, as specified above in Task 7, at the same time as the web-ready and print ready version of the second follow-up report.

Deliverables:

Draft second follow-up report

Draft final second follow-up report

Final second follow-up report Second follow-up data file, data analysis plan, and codebook

III. SCHEDULE OF DELIVERABLES

The contractor shall submit one copy of each final deliverable to the CO; all other deliverables are submitted to the COR. The contractor shall submit an electronic copy of all deliverables to the COR unless otherwise specified.

Task 1—Meeting with ED (FFP)

Task 2—Technical Working Group for the Evaluation (FFP)

Task 3—Develop Sampling Plan (FFP)

Task 4—Revise Study Design (FFP)

Task 5.1—Development of Data Collection Forms (CPAF)

Task 5.2—Prepare Baseline OMB Package Materials (FFP)

Task 5.3—Prepare Follow-Up OMB Package Materials (FFP)

Task 6—Data Collection (CPAF)

Task 7—Data Analysis and Report Preparation (CPAF)

Task 8—Reporting Requirements (FFP)

Option #1: Second Follow-Up (2016-2017) Data Collection

Task 9—Communicate with ED (FFP)

Task 10—Technical Working Group (FFP)

Task 11—Data Collection (CPAF)

Task 12—Data Analysis and Report Preparation (CPAF)