Sources_Sought-_Impact_Evaluation.pdf

IMPACT EVALUATION TEACH AND SCHOOL

LEADERSHIP INCENTIVE PROGRAM

FY18 ACQUISITION

Tracking No.:IES180050AP

Request for Information & Market Research

Facilitated by the Office of the Chief Financial Officer, Contracts and

Acquisitions Management Office

Contents Introduction

Contact Information

Request for Capability Statements

Vendor Questions

List of Attachments

Draft Performance Work Statement

Introduction

In order to support the FY18 competition, U.S. Department of Education ( the “Department”) announces an extensive Request for Information and Vendor Engagement effort. The

Department will issue a Request for Proposal (“RFP”) in FY18 to contract with a vendor, or team of vendors to determine if sufficient interest and capability exists in the marketplace for the

Impact Evaluation to Inform the Teacher and School Leader Incentive Program (“TSL evaluation”). Please read this document in its entirety. The information that you provide may play a substantial role in shaping this procurement and may affect the composition of the

Performance Work Statement (“PWS”), Instructions to Offerors, and Source Selection Criteria.

Vendors are encouraged to provide feedback now so as to enable the Department to address as many questions as possible before release of the RFP.

This document consists of three major information request components to be submitted in three parts:

(1) Request for capability statements.

(2) Targeted vendor outreach questions and responses.

(3) Request for comment and feedback on the PWS.

Detailed information for each request is located in the appropriate subsection below. In the event that vendors have questions, please note the contact information for Jaamal Jennings and Ryan

Battad are listed below.

Please review this document carefully as it is designed to streamline the acquisition process and avoid an extensive question-answer period during the active procurement phase. Please feel free to submit additional questions as part of your document response so that the Department may address your questions prior to release of the solicitation.

Please note that all RFP-related documents in this RFI are draft versions. The content of each document may change substantially between release of this RFI and release of the final RFP. The final RFP will be governed by the terms, conditions, instructions, and criteria in that document.

Offerors should not rely on the contents of this RFI for preparing final proposals. To the extent that information provided in this document is inconsistent with or contradicts information contained in the final RFP, the RFP controls.

Submission Instructions

I. Formal RFI Response

Detailed information for the required submission is provided below. Offerors should submit, via email, the following documents to Jaamal.Jennings@ed.gov with a cc to

Ryan.Battad@ed.gov no later than May 2, 2017 at 11:00am.

file://///WDCFB6FPR03/Groups/OCFO/CAM/All%20Contracts/Working/FY15%20-%20FY19/2018%20Solicitations/IESE180002AP%20-%20ERIC/Draft/Jaamal.Jennings@ed.gov mailto:Ryan.Battad@ed.gov

I. One PDF document that contains your Capability Statement (PDF).

II. One PDF document that contains your response to vendor questions.

III. One WORD document that contains any comments, track change revisions, or edits to the DRAFT PWS/Instruction to Offerors/Evaluation for Award.

Narrative justifications for changes are also welcome if appropriate. If offerors have additional questions regarding any of the three documents please feel free to include this information in a separate heading in your WORD document that contains these questions in addition to any narrative information.

Offerors should submit all three documents in one email, not to exceed ten (10) pages.

Please note that the Department imposes a 15MB file size limitation. This limitation accounts for potential file size increases associated with Microsoft Cloud services and may be different from earlier Department file size restrictions.

Participation in this vendor outreach effort is entirely voluntary, but it is an opportunity to let the Government know how it should develop its acquisition strategy. A response to this RFI is not required in order to respond to any solicitation that the Department issues.

Contact Information Please note that only the individuals listed below are authorized points of contact for this competition. No members of the IES staff (project staff or otherwise) will respond to inquiries even if the individual is responsible for technical direction under an existing impact evaluation contract.

The following members comprise the acquisition team for this procurement:

Primary Points of Contact

Jaamal Jennings

Jaamal.Jennings@ed.gov

(202) 245-6499

Contract Specialist

Ryan Battad

Ryan.Battad@ed.gov

(202) 245-6527

Contracting Officer/Source Selection Authority

Dissemination of Procurement Information

Interested vendors are strongly encouraged to communicate via email. The Department does not intend to post a Questions and Answers amendment to this notice. Instead, the Department intends to take those questions as items that will be utilized to strengthen the Department's

RFP/Solicitation documents.

Respondents may be contacted for additional information and/or invited to discuss your approach/questions in person or via teleconference. A letter stating your interest or requesting a copy of the solicitation is NOT sufficient to determine contractor capability. All announcements regarding the procurement will be posted to FedBizOpps.gov.

mailto:Jaamal.Jennings@ed.gov mailto:Ryan.Battad@ed.gov

Request for Capability Statements

Proposed NAICS Information: 541611, Administrative Management and General

Management Services; and 541720 Ressearch and Development in the Social Sciences and

Humanities

Proposed Product Service Code: R499 Other Professional Services; AF11 R&D:

Educational – Basic Research; AF16 R&D: Educational-Management and Support; B542

Educational Studies

The Department requests capability statements in order to evaluate what, if any, set aside determinations are appropriate. We reserve the right to use other resources and information to inform our acquisition strategy.

The Department will not review generic capability statements because generic capability statements tell us nothing about your ability to execute an impact evaluation contract. Rather, we will only review capability statements that conform to the specifications provided below.

Format and Instructions for Capability Statements

Capability statements are limited to 10 pages (not including the cover page) and must be submitted in PDF format. The Department will not consider material in excess of 10 pages.

1. Cover Page containing the following information:

Name of Company

Primary Point of Contact (phone and email)

DUNS Number

Small Business Status for the applicable NAICS code (if not a small business, please indicate “OTSB” in this field)/ Please also include all socioeconomic categories for your company and the associated NAICS codes

Number of Employees

Gross Annual Receipts

GSA Holder? If yes, GSA Contract number

Are you a Service-Disabled Veteran Owned Business (Yes/No)

Are you a certified HubZone Small Business (Yes/No):

Are you a Woman Owned Small Business (Yes/No):

2. Substantive Content

The capability statement must describe the collective expertise and experience of your company’s personnel, in each of the following areas. Clearly indicate if the expertise is not possessed by the prime contractor, but instead by a potential subcontractor or affiliate:

Experience and skill in managing, designing and conducting large-scale random assignment studies of interventions designed to improve educational outcomes.

Interested parties shall provide a minimum of two examples of personnel’s successful past experience and skill in leading, managing, designing, and conducting all aspects of evaluations (preferably in education settings) that are similar to this procurement in content (e.g., professional development, coaching/mentoring), methodology (e.g., randomized controlled trial), complexity (e.g., recruitment effort, intervention features, implementation support), scope (e.g., national), sample size, and data collection modes

(e.g., district administrative records, principal, teacher, and grantee surveys).

Experience and skill in authoring well written, policy relevant publications with clear take-home messages for wide audiences. Interested parties shall provide a minimum of two examples of personnel’s involvement with authoring publications that clearly summarize experimental findings and present them in a manner such that they can be understood and used by a wide audience.

Substantive knowledge relevant to the DRAFT TSL evaluation, including teacher professional development, mentoring/coaching, or other methods of relevant teacher support across geographically dispersed, multi-site settings. Interested parties shall provide a minimum of two examples showing that personnel have relevant knowledge to effectively study and support implementation of the intervention, particularly capitalizing on what is known from the research literature.

In addition, the capability statement shall address the following experience and capacity at the corporate level:

Experience with projects similar in nature and scale to that described in the Draft

TSL evaluation PWS. Interested parties shall provide evidence of successful corporate experience with a minimum of two past projects that are similar to that described in the Draft TSL evaluation PWS, in terms of the specific tasks to be performed, scale, and scope required.

Vendor Questions

(1) What, if any, additional information should the Department provide in order to enable you to accurately estimate the price/cost of each task? How do you propose to revise/mitigate these issues?

(2) Are the NAICS and PSC codes appropriate for this procurement? If not, please justify an alternative.

(3) Can you identify any barriers to competition? If so, please identify and describe how the

Department could reduce or eliminate those barriers.

(4) What level of collaboration do you expect from the Department?

(5) Do you agree with the designation of for each task/subtask in the PWS? If not, please provide a justification for changing the designation for each task or subtask.

(6) Do you agree with the contract structure, 49-months with overlapping periods of performance, outlined in the PWS? If not, please provide an outline and justification for a different structure and/or how the task deliverables should be structured so that it can align with each period of performance.

(7) Does your organization presently hold a relevant GSA contract or other Government

Wide Acquistion Contract (GWAC)? If yes, please provide the contract number. Also, what pricing/contract types does your GWAC allow?

(8) If you are a small business who believes they can perform on this contract as a prime, please provide the following information:

o Gross annual revenue;

o Number of employees;

o Value of contract(s) of the similar contract (Total contract value); and o Point of contact(s)for the effort being identified

(9) Please provide a rough order of magnitude (ROM) on a reasonable fee percentage for this effort and the rationale behind it, to include identifying risks, problems, etc. As of right now, the Government is estimating an 8% fee. Please note their is no guarantee the

Government will adjust the fee percentage as result of this RFI, and it ultimately reserves the right to set the fee percentage it deems is in the best interest of the Government.

(10) Can this effort be acquired using other than cost type contracts (i.e. Time and

Materials, Labor Hour, etc)? If so, please provide additional details on your suggested pricing structure and how it is in the best interest of the Government to consider such a contract pricing structure

(11) Is there any other information that you wish to convey to the Department?

List of Attachments

Draft Performance Work Statement with Appendicies

Draft Performance Work Statement U.S. Department of Education

Institute of Education Sciences (“IES”)

National Center for Education Evaluation and Regional Assistance

An Impact Evaluation to Inform the Teacher and School Leader Incentive Program

INTRODUCTION

The National Center for Education Evaluation and Regional Assistance (“NCEE”) of the U.S.

Department of Education’s (the “Department”) Institute of Education Sciences (IES) requires a contract to conduct an impact evaluation of a core strategy that is common among the funded

Teacher and School Leader Incentive Fund (TSL) program FY2017 grants. In addition, to the conduct of an impact evaluation, the contract will also collect data to provide information about the strategies funded by TSL 2017 grants and for the program’s performance indicators.

Period of Performance includes: one 16-month base period and three option periods that would result in a 49-month contract in total.

The base contract includes recruitment of study schools to participate in the impact evaluation as well as the development of data collection forms and development of the OMB forms clearance package, a TWG meeting (with design memo), and recruitment of teacher leaders. Based on documentation of successful study sample recruitment, the Department will decide whether or not to exercise Option Period I by September 2019.

Option Period I includes intervention implementation (teacher leader role strategy) and data collection during the 2019-20 school year. Option Period I is 17 months and overlaps with the base contract by 4 months. Based on documented successful conduct of activities during school year 2019-20, ED will decide whether or not to exercise Option Period II by April 2020.

Option Period II includes data collection including administrative data, data analyses, a technical working group meeting, and a memo to provide the program office with information about the

TSL GPRA measures, and initial draft report writing.

Option Period II is 23 months and overlaps with the Option Period I by 10 months. Based on documented successful conduct of Option period II study activities, ED will decide whether or not to exercise Option Period III by March 2021.

Option Period III is 19 months and overlaps with the Option Period II by 12 months.

Option Period III includes a grantee survey to collect GPRA data, a second GPRA memo, and final draft report preparation and revisions through IES’s standards and review office (SRO) clearance process.

Base Option Period I Option Period II Option Period III

Sept. 2018 -

Dec. 2019

Sept.2019 - Jan. 2021 Apr. 2020 – Feb. 2022 Mar. 2021-Sept 2022

16 months 17 months 23 months 19 months

Overlaps with base by 4 months

Overlaps with Period I by

10 months

Overlaps with Period II by 12 months

Months 1-16 Months 13-29 Months 20-42 Months 31-49

A. Legislative Authority

TSL is authorized in the Elementary and Secondary Education Act of 1965 as amended through

P.L. 115-64. The Director of IES is directed to conduct an independent evaluation to measure the effectiveness of the program. Further, the evaluation must “measure the effectiveness of the program in improving student academic achievement, the satisfaction of the participating teachers, principals, or other school leaders, and the extent to which the program assisted the eligible entities in recruiting and retaining high-quality teachers, principals, or other school leaders, especially in high-need subject areas.”

B. TSL Program

The TSL program assists States, local educational agencies (LEAs), and nonprofit organizations to develop, implement, improve, or expand comprehensive performance-based compensation systems or human capital management systems for teachers, principals, and other school leaders

(especially for teachers, principals, and other school leaders in high-need schools) who raise student academic achievement and close the achievement gap between high- and low-performing students. The FY2017 notice inviting applications contained four absolute priorities and two competitive preference priorities (see federal notice, https://www.gpo.gov/fdsys/pkg/FR-2016-12-

20/pdf/2016-30643.pdf).

A major objective of the program is to improve educator effectiveness through the use of human capital management systems that measure performance and guide ongoing, targeted support.

The Every Student Succeeds Act (ESSA) includes a mandated independent evaluation of the program that measures the effectiveness of the program in:

Improving student academic achievement;

The satisfaction of participating teachers, principals, or other school leaders; and

Improving recruiting and retaining high-quality teachers, principals, or other school leaders.

C. Key Research Questions

One goal of this evaluation is to provide implementation information about the activities funded through the FY2017 TSL grants. A second goal is to study the effectiveness of a commonly funded component of TSL grants. Because the grantees have already been awarded, the impact evaluation aspect of the study may be conducted in non-TSL grantee districts or schools. The evaluation report will include both a description of TSL grantee schools and activities as well as https://www.gpo.gov/fdsys/pkg/FR-2016-12-20/pdf/2016-30643.pdf https://www.gpo.gov/fdsys/pkg/FR-2016-12-20/pdf/2016-30643.pdf effects of the commonly funded component. The report shall link the impact evaluation results as much as possible to TSL funded activities for the purpose of program improvement.

A review of TSL grants awarded in FY2017 indicates that the grants include multiple strategies across a wide array of strategies, with the most common strategy being funding for teacher leadership roles. The teacher leadership role typically offers additional compensation to highly effective teachers to take on a leadership role that includes working with their lower-performing colleagues to improve teaching. The evaluation shall include both impact and implementation information to address the following research questions:

1. What is the impact on student achievement of funding teacher leader roles? What is the impact of this strategy on educator satisfaction, recruitment, and retention?

2. What are the experiences and challenges of districts and educators when implementing this strategy, both within the impact study schools and FY2017 TSL grant funded schools?

3. What are the implementation experiences of FY2017 TSL grantees?

The impact evaluation shall include voluntary districts (not TSL grantees) willing to implement a teacher leader role strategy and participate in an impact evaluation. To help make the evaluation results relevant to the FY2017 TSL grants, the contractor shall recruit schools similar in characteristics to TSL grantee schools that agree to implement the teacher leader strategy. In addition, the specific teacher leader strategy studied shall be similar in nature to that typically implemented by TSL grantees. The evaluation shall include data collection of TSL grantees as well as within districts and schools participating in the impact evaluation.

D. Evaluation Approach and Implementation of Teacher Leader Role

The purpose of ED’s evaluation is to evaluate the effectiveness of teacher leader roles similar to those funded by the TSL program and to describe TSL grants. The contractor shall consider the following assumptions:

ED awarded 14 FY2017 TSL grants (9 of which proposed a teacher leader role strategy: see https://innovation.ed.gov/what-we-do/teacher-quality/teacher-and-school-leader-incentive-program/awards/).

The impact study shall include approximately 10 districts and 100 schools. Recruited schools shall be high-need, consistent with eligibility focus for TSL grants.

Within each of the participating districts, schools shall be randomly assigned to treatment

(high-performing teachers within each assigned treatment school shall be identified and recruited by the contractor to receive payment in exchange for providing support to 2-3 low-performing teachers in their school) or control (these schools do not use teacher leaders to support their low-performing teachers).

The contractor shall provide teacher leaders with an orientation session prior to the start of the 2019-20 school year that includes expectations about their teacher leader role. In addition, the contractor shall provide minimal implementation oversight (to ensure that selected teacher leaders follow through with expected teacher leader role activities).

https://innovation.ed.gov/what-we-do/teacher-quality/teacher-and-school-leader-incentive-program/awards/ https://innovation.ed.gov/what-we-do/teacher-quality/teacher-and-school-leader-incentive-program/awards/

Up to two teacher leaders per school.

Data collection for the impact study shall consist of teacher leader enrollment and activity forms, a teacher survey for the implementation year (background, support experiences and professional development activities and hours), and a principal survey, student records data for students of the teacher leaders and the teachers in the grade in which they serve as a teacher leader, and a grantee survey (to broadly describe the strategies funded by TSL and to provide more in-depth information about teacher leader role activities and challenges for those grantees implementing the strategy). The contractor shall administer the grantee survey in spring 2020 and spring 2021.

To provide information about the TSL grantees, data collection shall also include a grantee survey (with data relevant for the program’s performance indicators, see https://www.gpo.gov/fdsys/pkg/FR-2016-12-20/pdf/FR-2016-12-20.pdf, p. 92804). To fully address the study’s mandate, relevant questions from the teacher and principal surveys shall be administered within the FY2017 TSL grantee schools.

E. Design Issues

Discussed below are a number of factors that will affect the evaluation design, and successful execution of the study and presentation of study findings. The contractor shall be cognizant of these issues while carrying out the evaluation.

District and School Context. Districts with a context similar to that required in TSL are most applicable for this evaluation. Schools that meet TSL eligibility requirements are also most applicable for this evaluation. Thus, the contactor shall address in the study design and site recruitment aspects of the proposal.

Counterfactual. Schools are likely to use multiple strategies to improve teacher performance.

Thus, the evaluation design and data collection shall address how this complicates the study service contrast and thus the design and interpretation of the impact evaluation.

Method of Identifying Teacher Leaders. The qualifications and skills of the recruited teacher leaders are likely to be a key factor in interpreting the impact results and replicating the strategy. The specific method used by TSL grantees is also important in making the results relevant to the TSL program effectiveness. Thus, the contractor shall address in the study design and site recruitment aspects of the proposal.

Teachers Targeted for Support. Who is supported and how are also likely to be key factors in using the impact results to inform the TSL program. Thus, the contractor shall address in the study design and site recruitment aspects of the proposal.

Survey Sample. The contractor shall collect relevant data for both impact study participants

(teachers and principals) as well as TSL grantees to meet the study goals while minimizing data collection costs.

https://www.gpo.gov/fdsys/pkg/FR-2016-12-20/pdf/FR-2016-12-20.pdf

Power analyses. The contractor shall calculate the appropriate sample size to achieve the targeted effect size of 0.10-0.12. The contractor shall fully explain and justify clustering as well as other power calculation assumptions.

SCOPE OF WORK

This section specifies the tasks and subtasks that the contractor shall perform.

Task 1—Opening Meeting with ED

The contractor’s project director and two additional key project staff members shall meet with the COR and other appropriate ED staff within ten (10) work days after the effective date of the contract award to discuss details regarding the tasks outlined in the proposal, scheduling activities, and other issues related to the contract. This meeting shall be at ED's offices in

Washington, DC. The contractor shall provide an agenda for the meeting within one week after award. The contractor shall come prepared to identify any areas of concern and to suggest ways of responding to these concerns. Within one (1) week following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each shall be addressed in the study.

Deliverables: Meeting Agenda

Memo summarizing meeting

Task 2—Prepare OMB package and Develop Data Collection Forms

The contractor shall prepare all relevant OMB forms and justification for a forms clearance package that includes all data collection instruments. While the preparation of the data collection forms occurs as part of the base contract, data collection using the forms will only occur if the option period(s) are executed. The package shall include a description of the study design, site recruitment, and data collection instruments, as well as the necessary forms required for OMB approval. The contractor shall ensure that the clearance package justifies the data collection activities and comprehensively responds to each required item in the OMB instructions. In addition, the contractor shall ensure that the forms clearance package includes: (1) the study mandate and objectives, (2) the research questions, (3) description of and justification for incentives, (4) description of sampling and recruiting procedures, and (5) limitations of the study and its potential use for policy decisions. The contractor shall submit a draft forms clearance package to the COR no later than December 2018. The COR will submit the accepted forms clearance package for clearance. Based on comments from internal and external clearance (after the first 60 day clearance process) the contractor shall make all needed revisions and submit a revised forms OMB clearance package for submission to OMB. The COR will then submit the accepted revised forms clearance package to OMB.

Data collection for the core study Option Periods I-III shall consist of:

Teacher Leader enrollment forms – The contractor shall construct forms that include background information about the teacher leaders and their prior experiences with coaching/mentoring within the impact study sample.

Teacher Leader activity forms – The contractor shall construct forms that provide information on the extent to which teacher leaders meet with teachers and a checklist of core supports provided within the impact study sample.

Teacher survey – The contractor shall construct a survey that provides data on teacher characteristics, the support activities that they receive, and their satisfaction with their level of support and the job more generally (the survey shall include teachers in the grades targeted by the studied strategy within the impact evaluation schools and a random sample of teachers in TSL grant schools). This survey shall be administered in the spring of the intervention year within the impact study sample as well as a sample of FY2017 TSL grantee funded schools.

Student administrative records data – The contractor shall collect administrative data on student characteristics and test scores, as well as other relevant covariate and subgroup information within the impact study sample. These data shall be collected for the year prior to the first implementation year (2018-19) and the year of implementation (2019-20).

Principal survey – The contractor shall construct a survey that provides data on principal characteristics, their satisfaction with teacher supports, and the extent to which their use of teacher leader roles helps with recruiting and retaining [high-quality] teachers, especially in high-need subject areas. This survey shall be administered in the spring of the intervention year within the impact study sample as well as a sample of FY2017 TSL grantee funded schools.

Grantee survey – The contractor shall construct a survey that provides information about the strategies funded within the 2017 TSL grants and provides more in-depth information about teacher leader role activities and challenges for those grantees implementing the strategy.

The grantee survey shall also collect information to construct the program’s GRPA performance indicators. The survey shall be administered in spring 2020. The contractor also shall administer the aspects of the grantee survey that address the program’s GRPA performance indicators in spring 2021.

The contractor shall design data collection instruments to address the key study questions. The contractor shall make use of or refine existing instruments to the extent possible and applicable.

For instruments where validity or reliability are not yet well established, the contractor shall pilot test each of the instruments with no more than nine respondents per instrument in no less than two schools to ensure that the questions can be answered in a reasonable amount of time, that the directions are easily followed, and that the contractor’s procedures and testing systems for administering the instruments are effective. Piloting is to be conducted prior to the final 60 days in the OMB submission process. The contractor shall ensure that instruments are revised as needed based on pilot test results.

The contractor shall coordinate its own Institutional Review Board (IRB) approval for the piloting, field testing, and full scale data collection with the IRB approval obtained by districts from their own institutions. Specifically the contractor is responsible for ensuring that all aspects of the program have been approved by the appropriate IRBs, including its own IRB.

Deliverables: Original submission OMB clearance package (first 60 days)

Revised OMB clearance package (second 60 days)

Final OMB clearance package (cleared package)

Documentation of IRB approvals

Task 3 - Selection and Recruitment of Study Districts and Teachers

During the selection and recruitment process, the contractor shall provide monthly selection and recruitment updates in the monthly progress reports (see Task 6).

The contractor shall, in consultation with the COR, develop a strategy for selecting study sites and prioritize potential sites based on criteria that the COR and contractor agree are pertinent to addressing the evaluation objectives. These shall include policy-relevant districts with capabilities sufficient to support the evaluation’s needs and a willingness to support and participate in the study.

Based on the exploratory contacts with potential sites and sample selection criteria developed in consultation with the COR, the contractor shall recruit study districts and teachers to participate in and cooperate with the study. The contractor shall secure final agreements to participate through signed memoranda of understanding so that a total of approximately 10 districts, 100 schools, and up to 100 teacher leaders are included in the study sample.

The contractor shall complete all district recruitment activities no later than May 2019. Random assignment of schools to the implement the intervention and selection of teacher leaders shall occur prior to the summer teacher leader orientation session in 2019.

As part of the recruitment process, the contractor shall, as appropriate, comply with: The Privacy

Act of 1974, P.L. 93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and

Privacy Act of 1974, 20 USC 1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41 CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the Federal common rule or ED’s final regulations on the protection of human research participants. The contractor shall also satisfy any state, district, or school policies regarding school participation in external research projects.

Deliverables: Memoranda of Understanding from districts.

Task 4—Technical Working Group

The role of the TWG will be to advise the contractor and ED on the conduct of the study including, but not limited to: study design, instrumentation, data collection and analysis, and draft study reports. The TWG shall be comprised of approximately 10 individuals, and include experts in teacher support and professional development, teacher quality, teacher policy, as well as experts in evaluation methodology. The contractor shall submit a list of proposed TWG members no later than 2 months after contract award. When preparing the list, the contractor shall evaluate the potential for conflicts of interest by assessing whether each potential TWG nominee has a bias or financial interest in the outcome of the study. After the COR approves the proposed TWG members, the contractor shall contact each proposed member to secure interest and intention to serve on the TWG.

Convene Technical Working Group. The contractor shall convene a meeting of the TWG in the base contract. The TWG meeting shall be held no later than 7 months after the contract award and shall focus on remaining design issues, data collection, and analysis issues. The contractor shall prepare a study design memo that refines the study design and incorporates a detailed description of the selected treatment to be studied. The study design document will be used as materials for informing the TWG of the study design prior to the TWG meeting. A draft of the study design document shall be provided to the COR 4 weeks prior to the TWG meeting.

After the meeting, the contractor shall submit a written summary of the meeting to the COR within one week, detailing suggestions and recommendations along with the contractor’s proposed actions. Proposed actions cannot go forward unless approved by the COR. The meeting site shall be arranged to minimize travel of Federal government staff members, with preference given to meeting in U. S. Department of Education facilities in Washington, DC. The contractor shall pay the travel, per diem, and honorarium expenses of the TWG members.

If TWG members recommend any significant deviation from the proposal, study design, instrumentation, data collection, or data analysis, the contractor shall submit the recommendation to the COR for approval prior to implementation. The contractor shall not make any significant changes in the study based solely upon the recommendation of TWG members without the written approval of the COR. The contractor also shall use the TWG members to review products, as appropriate, when group meetings are not required.

Deliverables: List of proposed TWG members

Study design memo

Summary of TWG meeting

Task 5.1— Implement and Monitor the Intervention Implementation

As part of the monthly reporting requirement (task 6), the contractor shall provide ongoing information about the intervention activities and as well as participant attendance information.

Fall implementation and monitoring. The contractor shall implement the teacher leader role intervention in the schools assigned to treatment. The contractor shall work with the study districts/schools to:

Arrange a half-day orientation for recruited study teacher leaders during the summer prior to the 2019-2020 school year.

Monitor implementation of the intervention in the fall of 2019.

Task 6- Reporting Requirements

In addition to the deliverables that are required for each task described above, the contractor shall submit one copy of the following performance reports each month to the CO and one copy to the

COR, simultaneously with the monthly invoice. The base phase includes performance reports for months 1 through 17.

Monthly Progress Report/Exception Reports. The contractor shall prepare monthly progress reports due at the same time as the monthly invoice. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, changes in personnel and their security designation, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

Monthly Manpower/Expenditure Reports. The contractor shall prepare monthly expenditure reports due at the same time as the monthly invoice. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

Deliverables: Monthly progress & manpower/expenditure report

OPTION PERIOD I

Task 5.2— Implement and Monitor the Intervention Implementation

As part of the monthly reporting requirement (task 8), the contractor shall provide ongoing information about the intervention activities.

Late fall-spring implementation and monitoring. The contractor shall continue implementing and monitoring the teacher leader role intervention during 2019-20 in the schools assigned to treatment.

Task 7.1—Data Collection

Using instruments approved by OMB, the contractor shall collect data as described in Task 2.

The contractor shall utilize appropriate techniques to obtain response rates of at least 85 percent according to the National Center for Education Statistical Standards

(http://nces.ed.gov/pubs92/92021.pdf). As part of the monthly reporting requirement (task 8), the contractor shall provide ongoing information about the data collection activities.

To ensure that data collected are of high quality, the contractor shall quickly identify patterns of inaccuracies and anomalies in the data (such as out-of-range variables, missing data, and unusual distributions of scores), such that immediate corrective action can be initiated. All such problems and subsequent corrections must be carefully documented and compiled into a technical report that shall be included with the final archived data set. The data monitoring activities shall be in place within 10 months after contract award. During data collection, the contractor shall include with the monthly progress reports updates on the progress of data collection based upon the data monitoring activities.

To ensure that the data collection and coding staff have the skills necessary to perform their duties, and to ensure consistency of training across time, the contractor shall design a comprehensive training program. The training shall document the minimum performance standards that the data collectors or coders must meet, the criteria by which the contractor shall ascertain whether data collectors and coders meet the performance standards, how often performance will be evaluated, and how the contractor will verify that the training of data collectors and coders will be consistent across all waves of data collection. All persons who collect, code, or otherwise have access to raw data must be trained in confidentiality issues and shall provide signed assurances of nondisclosure of confidential information. The contractor shall maintain these records and provide information on these procedures to the Government upon request. When possible, individuals collecting and/or coding data shall have no knowledge of each school’s treatment condition.

All persons who collect, code, or otherwise have access to raw data must be trained in confidentiality issues and shall provide signed assurances of nondisclosure of confidential information. The contractor shall maintain these records and provide them to the Government upon request. When possible, the contractor shall ensure that individuals collecting and/or coding data do not have knowledge of the study participant’s treatment condition. The contractor, and all subcontractors, shall comply with: The Privacy Act of 1974, P.L. 93-579, 5 USC 552 a; the

“Buckley Amendment,” Family Educational and Privacy Act of 1974, 20 USC 1232 g; The

Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41

CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the Federal common rule or the

Department’s final regulations on the protection of human research participants. The contractor shall be responsible for safeguarding any PII that is collected for the purposes of this contract.

The contractor must also comply with any relevant requirements of the Privacy Act, 5 U.S.C.

552a; the E-Government Act of 2002, 44 U.S.C. §208; as well as relevant OMB directives, such as OMB M-06-16 and OMB M-17-12. The Contractor shall abide by and follow all

Departmental privacy policies, procedures, processes, and standards. All electronically stored sensitive data shall be encrypted at rest and in motion. The contractor shall also satisfy any state, district, or school policies regarding school participation in external research projects. Contractor and subcontractor employees who “maintain” (collect, maintain, use, or disseminate) data shall comply with the requirements of the confidentiality standards in section 183 of the Education

Sciences Reform Act of 2002 (ESRA) (20 U.S.C. 9573).

Task 8- Reporting Requirements

In addition to the deliverables that are required for each task described above, the contractor shall

COR, simultaneously with the monthly invoice. The option period I phase includes performance reports for months 18 through 30.

activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, changes in personnel and their security designation, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, OPTION PERIOD II

Task 7.2 — Data Collection (con’t)

Using instruments approved by OMB, the contractor shall collect all study administered data

(teacher survey, grantee survey, administrative records) to estimate the impact of the intervention on the outcomes of interest and to answer the evaluation questions. The contractor shall utilize appropriate techniques to obtain response rates of at least 85 percent according to the National

Center for Education Statistical Standards (http://nces.ed.gov/pubs92/92021.pdf).

To ensure that data collected are of high quality, the contractor shall use all procedures outlined in task 7.1 above and provide monthly information about data collection activities as part of the monthly report in task 11.

Task 9.1—Data Analyses, GPRA memo, and Draft Report

The contractor shall conduct data analyses and prepare a draft report that clearly addresses the evaluation questions.

The contractor shall prepare a memo that provides information for the TSL program GPRA indicators 2 months after the completion of data collection.

The contractor shall ensure that report is written and organized, so that it is accessible to policy makers and research-savvy practitioners rather than academic researchers. The contractor shall include a stand-alone executive summary of the study’s findings.

The contractor shall submit a draft report no later than 4 months after the TWG meeting (Task

10).

Deliverables: Draft report

GPRA memo

Task 10—Technical Working Group

Convene Technical Working Group. The contractor shall convene a meeting of the TWG members to discuss the study findings. The meeting shall be held no later than April 2021.

After the meeting, the contractor shall submit a written summary of the meeting to the COR within one week, detailing suggestions and recommendations along with the contractor’s proposed actions. Proposed actions cannot go forward unless approved by the COR. The contractor shall pay the travel, per diem, and honorarium expenses of the TWG members, and any lodging required by the TWG members. Finally, the COR will secure a meeting space at

ED facilities.

The contractor shall provide copies of all reviews of products by the panel to the COR within five (5) work days of receipt by the contractor. If TWG members recommend any significant deviation from the data analysis plan, the contractor shall submit the recommendation to the

COR for approval prior to implementation. The contractor shall not make any significant changes in the study based solely upon the recommendation of TWG members without the written approval of the COR.

Deliverables: Summary of TWG meeting

Task 11—Reporting Requirements

In addition to the deliverables that are required for each task described above, the contractor shall submit one copy of the following performance reports each month to the CO and one copy to the COR, simultaneously with the monthly invoice. The option period II phase includes performance reports for months 31 through 44.

activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, changes in personnel and their security designation, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, OPTION PERIOD III

Task 9.2—Final Report and Data File

Within three weeks, the COR will review and provide comments on the draft report submitted under Task 9.1. The contractor shall submit a revised draft report ready for submission to the

Standard’s and Review Office (SRO) within 2 months of ED comments. ED will submit the draft report to the SRO. Within approximately 2 months, ED will provide comments to the contractor from the SRO. The contractor shall make final revisions based on the SRO comments and submit a revised final first report within one month. The contractor shall revise the report until all peer review comments are addressed and the report clears the SRO review (current experience is an average of 3 rounds of successively less extensive comments). The contractor shall provide a web-ready and a print ready version of the report to the COR within 2 weeks after the report is cleared by the SRO.

Deliverables: Revised draft report, SRO-ready

Final report

Print and web-ready report

Task 12 – Grantee GPRA data collection and memo

Using instruments approved by OMB, the contractor shall administer the GPRA indicator information questions from the grantee survey in the spring of 2021. The contractor shall utilize appropriate techniques to obtain response rates of at least 85 percent according to the National

Center for Education Statistical Standards (http://nces.ed.gov/pubs92/92021.pdf).

To ensure that data collected are of high quality, the contractor shall use all procedures outlined in task 7.1 above and provide monthly information about data collection activities as part of the monthly report in task 11.

The contractor shall prepare a memo that provides information for the TSL program GPRA indicators 2 months after the completion of data collection.

Deliverables: GPRA memo

Task 13—Reporting Requirements

In addition to the deliverables that are required for each task described above, the contractor shall

COR, simultaneously with the monthly invoice. The option period III phase includes performance reports for months 45 through 49.

activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from ED. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, INFORMATION TECHNOLOGY AND INFORMATION SECURITY REQUIREMENTS

T…