DRAFT PWS_IDIQ and Task Order 1.docx

U.S. Department of Education Institute of Education Sciences National Center for Education Evaluation and Regional Assistance

DRAFT Performance Work Statement

EVALUATING FAMILY ENGAGEMENT AND CHOICE

Tracking number: IESE200278AP Solicitation: 91990020R0018

INTRODUCTION

The U.S. Department of Education’s (Department) Institute of Education Sciences (IES) requires at least one contractor to assist IES in conducting evaluations of the Department’s efforts to improve families’ engagement in education. Family engagement is central to the Department’s mission as signaled by the 2018-22 Strategic Plan[footnoteRef:2], new grant programs, and new priorities for existing programs. [2: Objective 1.1: “Increase high-quality educational options and empower students and parents to choose an education that meets their needs”: https://www2.the Department .gov/about/reports/strat/plan2018-22/strategic-plan.pdf, ]

This Indefinite Delivery-Indefinite Quantity contract (IDIQ) is intended to help IES efficiently describe the implementation of Department-funded programs and examine the effectiveness of strategies to improve them. By awarding a contract to at least one contractor with appropriate content knowledge, data collection, and methodological experience to undertake all of the tasks described below, IES expects to maximize the benefits of what can be learned in a cost-effective manner. This will occur through developing and fielding surveys that gather consistent information across programs, identifying common engagement strategies, and further linking across studies to reduce the time and costs that would otherwise be involved in starting up individual contracts for evaluation of each program separately.

The IDIQ shall have an ordering period of 60 months. All task orders shall begin within the 60-month ordering period, but may have a duration that extends for up to 24 months beyond the end of the ordering period. Task orders may be issued as fixed price, Time and Materials, or cost reimbursement contracts. Each task order will include one or more tasks related to carrying out a specific evaluation or an aspect of an evaluation. The scope of each task order will depend on IES’ needs, including which activities are appropriate for each desired evaluation. For example, some task orders may include only design work and some task orders will include external data collection activities while others will not.

THE EVALUATIONS

A. Legislative Authority

The Education Sciences Reform Act of 2002 established IES as the unit within the Department to evaluate federal education programs and practices.

B. Background

Family engagement is a term that covers a number of kinds of family interaction with the education system. It can mean parenting, communicating, supporting school, learning at home, decision-making, and collaborating with the community. Prior research has linked engagement to higher student grade-point averages and scores on standardized tests; enrollment in more challenging academic programs; passing more classes and earning more credits; better attendance; improved behavior at home and at school; and better social skills and adaptation to school.[footnoteRef:3] Although the size of impacts differs by the kind of engagement and grade level, overall, this research suggests family engagement is one of the strongest predictors of children’s success in school and beyond.[footnoteRef:4] [3: Henderson, A. T., and Mapp, K.L. (2002). A new wave of evidence: The impact of school, family, and community connections on student achievement. Austin: National Center for Family and Community Connections with Schools. (Update forthcoming.) ] [4: Van Voorhis, F.L., Maier, M. F., Epstein, J. l., and Lloyd, C.M. (2013). The impact of family involvement on the education of children ages 3 to 8: a focus on literacy and math achievement outcomes and social-emotional skills. New York: MDRC.]

The Department funds multiple programs focused on improving families’ engagement in education (e.g., Statewide Family Engagement Centers, Parent Training and Information Centers for families of students with disabilities, and Indian Education Demonstration Grants, among others). While these programs share a focus on family engagement, they emphasize different approaches. Some programs prioritize dissemination of information meant to engage families in school choice and others prioritize information related to education related rights. Some programs emphasize building statewide capacity for family engagement and others serving parents directly. Therefore, it would be useful to better understand how the different programs carry out their work, the challenges faced by grantees, and the needs for rigorous and up-to-date evidence regarding effective strategies. The goal of this contract is to help programs carry out their work and to potentially help them develop and test engagement strategies.

All deliverables shall be submitted electronically to the COR. The contractor shall also electronically submit one copy of each monthly report and final deliverables to the CO.

Task 1—Meetings with the Department

1.1 Initial Meeting. The contractor’s project director and two other key project staff members shall meet with the Contracting Officer (CO) and/or Contracting Specialist (CS), the IES Contracting Officer’s Representative (COR), and other appropriate Department staff within 10 business days after the effective date of the contract award or award of a task order to discuss details regarding the tasks, scheduling activities, and other issues related to the contract or a particular task order. The meeting will be held in the Department’s office in Washington, DC. The contractor shall come prepared to identify any areas of concern and to suggest ways of responding to them. Within two weeks following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each shall be addressed as part of the contract.

1.2 Additional Meeting. The contractor shall meet with the COR once in Washington, DC to discuss the progress of contract activities, including all current task orders, and any issues or anticipated problems and proposed solutions to those problems. Within two weeks following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each shall be addressed as part of the contract.

Deliverable: Memo summarizing meetings with the COR

Task 2—Technical Working Group (TWG)

2.1 Establish TWG. The role of the TWG will be to advise the contractor on an individual task order or collection of task orders that address a similar topic. Under this contract there may be a single TWG or multiple TWGs, depending on IES’ needs. Each TWG panel shall be comprised of up to eight individuals that, together, have expertise as specified in the task order(s). The contractor shall evaluate the potential for conflicts of interest by assessing whether any individual considered for inclusion on the TWG has a bias or financial interest in the outcome of the study or study tasks. The contractor shall have TWG members complete conflict of interest forms before appointment to the TWG.

The contractor shall submit a list of potential TWG members to be reviewed and approved by the COR within eight weeks of task order award. The contractor shall contact each approved member and submit letters of commitment and documentation of consulting rates to the COR four weeks after COR approval of the TWG list.

2.2 Convene In-person TWG Meeting: The first meeting of a TWG shall be held in person in Washington, DC within eight months of task award. The first TWG meeting will be an opportunity both for the panel members and the contractor team to get to know each other and for discussion of the design of the study or study tasks specified in the task order. Subsequent meetings of a TWG can be in-person or virtual (see Task 2.3). Each in-person meeting will take place in Department facilities in Washington, DC. The contractor shall cover the travel, per diem, and honorarium expenses of the TWG members.

Four weeks prior to the meeting, the contractor shall provide to the COR a draft agenda, a draft slide deck, and a plan for a guided discussion of key topics and issues. Within two weeks after the meeting, the contractor shall submit a brief but detailed summary of the key observations and recommendations provided by TWG members, along with the contractor’s proposed actions. The contractor shall not make any significant changes in the task order activities based solely upon the recommendation of TWG members without the written approval of the COR and CO.

2.3 Convene Virtual TWG Meeting. The contractor shall convene a 3-4-hour web-based meeting of a TWG. The date for the meeting shall be set in consultation with the COR. Four weeks prior to the meeting, the contractor shall provide the COR a draft agenda, a draft slide deck, and a plan for a guided discussion of key topics and issues. Within two weeks after the meeting, the contractor shall submit a brief but detailed summary of the key observations and recommendations provided by TWG members, along with the contractor’s proposed actions. The contractor shall not make any significant changes in the task order activities based solely upon the recommendation of TWG members without the written approval of the COR and CO.

Draft meeting materials Summaries of meetings Task 3— Feasibility and Design

The contractor shall undertake activities to assist IES in determining the key parameters of an implementation or impact study that might be carried out under one or more task orders.

3.1 Feasibility Assessment for Effectiveness Studies. In consultation with the COR, the contractor shall identify or support development of one or more policy-relevant family engagement strategies. The contractor shall then assess the feasibility of examining the/those strategies through a random assignment or regression discontinuity design. These strategies shall be particular services or approaches to family engagement that are allowable under and, preferably, newly emphasized in, one of the studied grant programs and have some suggestive evidence of effectiveness. To conduct the feasibility assessment, the contractor shall conduct telephone interviews with state or local grant coordinators as appropriate to obtain information about the prevalence of a particular strategy of interest and how it is implemented. If relevant, the contractor shall conduct telephone interviews with parents or other target recipients of the family engagement strategy to gather information on how the strategy is received and perceived. The contractor shall submit a list of proposed individuals for the interviews and a list of the questions that these individuals would be asked to the COR no more than 8 weeks after task order award. Once approved, the contractor shall carry out the interviews. As necessary, the contractor shall modify or develop variants of the strategy for eventual testing. Within 16 weeks of task order award, the contractor shall prepare a memo up to 10 pages long summarizing the design options and the contractor’s recommendations

3.2 Survey Sample Design. As needed, the contractor shall assist the Department in determining design options for one or more surveys with representative or opportunistic samples. In consultation with the COR, the contractor shall identify feasible and policy relevant stratification approaches and the number of local school districts, trainers, teachers, service providers, families, or other type of respondent needed to fulfill the objective of the survey, as specified by the COR. Within 12 weeks of being assigned this task by the COR, the contractor shall prepare a memo up to 10 pages long summarizing the sample design options and the contractor’s recommendations.

3.3 Interview/Focus Group Sample Design. As needed to gain more in-depth descriptive data about how and why grantee programs were created, the kinds of services offered, and families’ needs the contractor shall assist the Department in determining design options for one or more sets of interviews or focus groups. In consultation with the COR, the contractor shall identify feasible and policy relevant purposive sampling or opportunistic approaches and the numbers of grant coordinators, SEAs, tribes, grant partners, subgrantees, contractors and/or recipients of program services including families needed to fulfill the objective of the interview, as specified by the COR. Interviews shall consist of pre-specified response categories that can be tabulated, as well as of probes for capturing idiosyncratic elaborations or examples. Within 12 weeks of being assigned this task by the COR, the contractor shall prepare a memo up to 10 pages long summarizing the sample design options and the contractor’s recommendations.

3.4 Study Design Plan. The contractor shall develop a data collection plan for addressing the research questions for one or more studies, as specified by the COR. A plan shall describe the information topics, domains, and constructs that are necessary to achieve the objectives of the study and, at a high level, how that information might be obtained and analyzed. It shall also clearly indicate whether the study is primarily to be an implementation or effectiveness study, and whether, if an implementation study, it is to be primarily quantitative, qualitative, or mixed methods. Within 14 weeks of being assigned the task by the COR, the contractor shall submit a draft version of the plan called for. The COR will review and provide comments on the draft plan within 2 weeks. Within 4 weeks after receiving COR comments, the contractor shall make revisions and submit a final version of the plan that could be included in an Office of Management and Budget (OMB) Forms Clearance package.

3.5 Data Collection Instruments. As needed, the contractor shall develop study intake forms, surveys, questionnaires to gather feedback on parent engagement strategies, interview protocols, data extraction forms or instructions, outreach or recruitment letters, or other instruments, as specified by the COR, to be used in conducting a study under one or more task orders in this contract. These instruments shall minimize burden on respondents while ensuring that key data items are collected. Within 4 weeks of the COR approving the study plan under which data shall be collected, the contractor shall submit a draft version of the instruments called for. The COR will review and provide comments on the draft instruments within 2 weeks. The contractor shall make revisions and submit a final version ready for pre-testing and/or inclusion in an Office of Management and Budget (OMB) Forms Clearance package within 4 weeks after receiving COR comments.

Task 4—Systematic Evidence Review

The contractor shall conduct an evidence review following the procedures and standards used by the WWC, including the use of WWC-certified study reviewers. This shall include determining policy-relevant categories of family engagement strategies to examine; developing a review protocol, searching the literature to identify relevant studies; screening in eligible studies; coding eligible studies; assessing the methodological quality of each eligible study; analyzing the results of eligible studies that meet WWC standards or meet standards with reservations; and synthesizing the results across studies.

The evidence report that is the culmination of this task shall address the question of what is known about the effectiveness of family engagement strategies – either in one topic area or many, as specified by the COR -- in improving key outcomes. The report shall include a description of the methodology of the review and the state of the literature on program strategies (number of studies identified in the scan for particular outcome measures, the number that meet WWC evidence standards). Most importantly, the report shall summarize the evidence from the studies meeting WWC standards and meeting WWC standards with reservations and identify gaps in the literature.

4.1 Create an Evidence Review Plan. Prior to conducting the review, the contractor shall submit a proposed plan for conducting the evidence review. The evidence review plan shall include parameters for the systematic evidence review, such as databases and other sources that the contractor will consult, search terms the contractor will use, and any limitations that will be imposed (e.g. in terms of publication years). The evidence review plan shall also include the contractor’s plans for systematically compiling the evidence base in an evidence report (e.g., report outline, proposed tables shells, intervention summary sheets, appendices).

The contractor shall submit the draft evidence review plan to the COR on a timetable specified in the task order. The COR will provide comments on the draft evidence review plan within two weeks. The contractor shall make revisions and submit a final evidence review plan to the COR.

4.2 Conduct the Review. Using the final evidence review plan, the contractor shall conduct the review and submit a draft evidence review report. The report length is expected to be no more than 15 pages, with information on the technical aspects of the review and full documentation of the interventions, data, analytic methods, and results of studies included in appendices. The contractor shall also include a one-page summary of the evidence report that contains enough information to form the basis for a press release.

The COR will review the draft report within three weeks. The contractor shall incorporate revisions to address COR comments and submit a revised report fully responsive to COR comments within four weeks after receiving COR comments. The COR will submit the revised evidence review report to IES’ Standards and Review Office (SRO) for review. The COR will provide comments from peer review to the contractor within approximately two months. The contractor shall be responsive to SRO’s comments and revise the report until it clears SRO. The contractor shall provide a web-ready (i.e. 508-compliant), final report to the COR within two weeks after the report is cleared by SRO.

Task 5 – Prepare OMB Forms Clearance Packages and Document Institutional Review Board Approval

The contractor shall prepare a forms-clearance package in order to receive approval from OMB for this effort. The contractor shall ensure that the clearance package follows the required OMB format, justifying the necessity for collecting data and comprehensively responding to each required item in the instructions, including: (1) the objectives of the study or collection of studies, (2) the research questions, (3) a crosswalk between the research questions and the information to be collected, (4) instructions to study participants, (5) description of and justification for incentives, (6) discussion of questions likely to be deemed “sensitive” by OMB, (7) limitations of the study(ies) and their potential use for policy decisions, and (8) table shells indicating how data will be tabulated and analyzed.

All forms clearance packages shall be brief, clear, and well written. Each package shall include the forms and instructions that will be used to obtain data or recruit respondents. Within 6 weeks of having a study design plan approved by the COR, the contractor shall submit a draft OMB package. The COR shall review and provide comments within two weeks and the contractor shall have two weeks after receiving comments from the COR to submit the final OMB clearance package. The COR will then submit the package to begin the OMB approval process. During the initial 60-day public comment period, the contractor shall pretest the data collection forms so that burden information can be updated in the final package. The contractor shall be prepared to make up to two revisions to forms and justification as required during the approval process.

The contractor shall obtain approval from its Institutional Review Board (IRB) for the evaluations conducted under any task order. The effort to obtain initial approval and to update approval as new task orders are issued should be conducted in conjunction with OMB clearance package efforts.

Task 6 – Outreach and Recruitment

The contractor shall develop and implement a strategy for identifying and recruiting participants (e.g., grant coordinators, grantee partners, SEAs, tribes, districts, curriculum developers, service providers/contractors, parents, teachers, students or others) to meet the sample size or comprehensiveness requirements specified in a task order. As part of the recruitment process, the contractor shall prepare notification letters and information packets for the prospective study participants. The notification packet shall include a non-technical brochure describing the study that is suitable for distribution to a broad audience of policy makers and educators. The contractor shall include in the letters and information packets general information on the study as well as specific information on the importance of the study, its purposes, products, timelines, and the expected benefits to be derived from participation in the study. Recruitment may occur through mailings, telephone calls, in-person visits, or discussion with potential participants at key conferences or meetings, as specified in the task order. If specified by the COR, recruitment may be conducted by a third-party that maintains opportunistic sets of survey respondents—e.g., standing panels of parents -- for the purpose of obtaining quick turn-around information on a key topic.

The contractor shall submit an outreach and/or recruitment plan, draft letters, and all other notification materials to the COR for review. Once approved by the COR, the contractor shall send the notification materials to potential participants and execute the recruitment plan. The contractor shall consult with the COR to finalize the list of participants for the study, and subsequently submit documentation of memoranda of understanding or other necessary participation agreements to the COR.

As part of the recruitment process, the contractor shall comply with: The Privacy Act of 1974, P.L. 93-579, 5 USC 552 a; the “Buckley Amendment,” Family Educational and Privacy Act of 1974, 20 USC 1232 g; The Freedom of Information Act, 5 USC 522; and related regulations, including but not limited to: 41 CFR Part 1-1 and 45 CFR Part 5b and, as appropriate, the Federal common rule or the Department’s final regulations on the protection of human research participants.

During recruitment, the contractor shall submit to the COR progress reports on a biweekly basis.

Deliverables: Recruitment plan/materials Progress reports on recruitment

Task 7— Intake, Random Assignment, and Monitoring for Effectiveness Studies

The contractor may need to set up a process for enrolling participants into a study (“intake,” including obtaining consent), for random assignment, and for monitoring implementation of an intervention.

7.1 Intake and Random Assignment. The contractor shall carry out intake at one or more specific points in time – e.g., at the beginning of each semester or of a school year -- or on a rolling basis over an extended period. Random assignment may be of families, students, teachers/classrooms, schools, or other units depending on the evaluation design and will be specified in the task order. Random assignment may be conducted by the contractor or in the field (at schools or other sites) to accommodate rolling intake. The contractor shall develop protocols and procedures to conduct intake and random assignment to be minimally disruptive to participants and program staff who may be involved in intake. During study intake and random assignment periods, the contractor shall provide biweekly progress updates to the COR.

7.2 Monitoring Implementation of Intervention. The contractor shall develop processes and procedures to support and monitor implementation of the intervention. During implementation, the contractor shall follow the COR approved processes and procedures to support high quality implementation. The contractor shall also use monthly reporting (see Task 10) to provide ongoing information implementation, including participant attendance information, if relevant.

Deliverables: Biweekly intake and/or random assignment updates Implementation monitoring and support protocol

Task 8—Data Collection

IES expects that studies conducted under these task orders will rely on a variety of data sources. These may include extant data (e.g., grant applications, annual performance reports and other grantee-collected data, program monitoring notes, prior program studies/analyses and current literature scans, local artifacts/documents); surveys of agency staff at the state or local level; family or student surveys or interviews; and state records, school records, or records from other agencies. The specific data to be collected or extracted shall be specified by task order. Interviews may take place by telephone or in-person depending on the program to be studied. During periods of survey or interview administration or data extraction, the contractor shall provide bi-weekly progress updates to the COR.

8.1 Administer Surveys to Grant Coordinators and Partners. The contractor shall administer the relevant survey to all program grant coordinators and partners (as applicable) as specified in each task order. The programs will provide lists of these individuals and letters of support. The contractor shall follow up with the program grant coordinators and partners through mail, email, or telephone calls in order to achieve a response rate of 100 percent.

8.2 Administer Surveys to Subgrantees and/or Contractors. After interviews with grant coordinators to confirm program structure (i.e., the types, number, and roles of subgrantees and/or contractors providing services), as approved by the COR, the contractor shall administer the relevant survey to the subgrantees and/or contractors. The contractor shall follow up with respondents through mail, email, or telephone calls in order to achieve a response rate of 85 percent.

8.3 Administer Surveys to Parents and/or Students. The contractor shall administer, if specified in the task order, the relevant representative survey to the local participants or ultimate beneficiaries in a family engagement implementation or effectiveness study, such as parents and students or recipients of technical assistance services (for example, local education agencies (LEAs), schools, or teachers), in the most efficient manner possible. The contractor shall follow up with respondents through mail, email, or telephone calls in order to achieve a response rate of 80 percent.

8.4 Conduct Interviews/Focus Groups with Grant Coordinators and Partners . As necessary to supplement surveys with information and examples that are idiosyncratic to each grantee, the contractor shall administer the relevant telephone/in-person interview to the program grant coordinators and partners (as applicable) as specified in each task order. In-person interviews may take place at opportunistic occasions such as project director meetings or conferences. The contractor shall follow up with the program grant coordinators and partners through mail, email, or telephone calls in order to achieve a response rate of 100 percent.

8.5 Conduct Interviews/Focus Groups with Subgrantees, Contractors, and/or LEAs. As approved by the COR to supplement surveys, the contractor shall administer the relevant interviews/focus-groups to a purposive sample of subgrantees and/or contractors in charge of carrying out grant activities. The contractor shall follow up with sampled respondents through mail, email, or telephone calls in order to achieve a response rate of 85 percent.

8.6 Conduct Interviews/Focus Groups with Parents and/or Students. The contractor shall administer, as specified in the task order, the relevant interviews/focus groups to a purposive sample of participants or the ultimate beneficiaries in a family engagement implementation study, such as parents and students, in the most efficient manner possible. The contractor shall follow up with respondents through mail, email, or telephone calls in order to achieve a response rate of 85 percent.

8.7 Conduct Site Visits. When appropriate and feasible, especially in studies where cultural norms make in-person interviews more likely to obtain candid information than surveys or telephone interviews, the contractor shall arrange for travel and expenses for contractor staff to travel to grantee site(s) to conduct certain data collection in person (case studies). Staff may, for example, spend 2-3 days observing the planning of parent engagement activities, collecting artifacts (e.g., service provider documents, printed outreach brochures, materials requiring some contextual information or on-site explanation) from the site illustrating the kinds of activities funded by the grant and, where possible, observing engagement activities and meetings taking place during the time of the visit.

8.8 Extract Data. In addition to extracting data from grant documents (e.g., grant applications, annual performance reports, etc.), the contractor shall obtain records and extract relevant data from designated subgrantees and/or contractors, such as service providers; such data may be in the form of data files or other information sources, such as project websites. The contractor shall obtain records from students in a parent engagement effectiveness study, as applicable, using the most efficient process possible. Student records may be obtained from state data systems (e.g., states’ longitudinal data systems) or from schools directly or other agencies (e.g., districts or SEAs) and include some combination of students’ demographic characteristics and outcomes targeted by the engagement intervention (e.g., literacy achievement, course taking, grades, attendance, or attainment), as specified in the task order. The contractor shall follow up to achieve a response rate of 85 percent.

8.9 Obtain Third Party Data. The contractor shall set up a Memo of Understanding with third party vendors, such as the National Student Clearinghouse (NSC) and obtain outcome data such as student postsecondary educational enrollment and attainment, as specified in a task order.

8.10 Access Other Administrative Records. The contractor shall assist IES in setting up a Memo of Understanding with another agency or organization in order to obtain data on services or outcomes for parents and/or students in a family engagement study, as specified in a task order. The contractor shall specify the data to be extracted and, if the data will be transferred to the contractor, arrange for secure transfer, or use the data on site at the other agency/organization.

8.11 Obtain NCES Restricted Use File data. The contractor shall obtain a NCES restricted use license and the data to conduct extant data analysis (for example, on nationally representative family and student surveys) as specified in a task order.

Biweekly progress updates on survey and interview administration and records collection

Task 9—Data Analysis and Reporting

9.1 Analysis Plan. The contractor shall develop a plan that describes the study design, if relevant, and identifies the data sources, key measures, and intended data analyses that will be included in an evaluation report, other external product, or internal memo. The plan shall include table shells and, if relevant, the statistical power for measuring impacts on the planned outcomes with the achieved sample size. The plan shall specify any sensitivity tests the contractor plans to conduct. In addition, when a primary data source is interviews or focus groups, the plan shall describe the methods for ensuring collection and analysis consist of key information with pre-specified response categories that can be tabulated, as well as of probes for capturing idiosyncratic elaborations or examples. The contractor shall submit a draft analysis plan to the COR as specified in the task order. The COR will provide feedback within two weeks and the contractor shall submit a final data analysis plan within two weeks of obtaining the feedback.

9.2 Study Report. Once the analysis plan developed under task 9.1 is approved, the contractor shall submit an outline for the report where the analyses will be presented. The COR will provide feedback within two weeks. The contractor shall conduct data analyses and prepare a report describing what is being studied and the results of the analysis. The contractor shall borrow the introduction, study design description, and data sources from the final analysis plan, rather than developing this from scratch, to the extent that it meets the COR’s approval.

The contractor shall make sure that the report is written and organized so that it is accessible to policy makers and research-savvy practitioners rather than academic researchers. The report text length is expected to be no more than 15 pages, with information on the technical aspects of the study included in appendices. To the extent feasible, the report shall include visually intuitive figures and graphics rather than tables.

The contractor shall also include a one-page “highlights” document. This shall be a stand-alone introduction to the report and contain enough information to form the basis for a press release.

The contractor shall submit a draft report to the COR on a date specified in the task order. The COR will review the draft report within two weeks. The contractor shall have two weeks to make revisions and submit a revised report, ready for IES’ peer review process overseen by the Standards and Review Office (SRO). The COR will submit the revised report to SRO for review. The COR will provide comments from SRO to the contractor within approximately two months. The contractor shall be responsive to SRO’s comments and revise the report until it clears SRO. Taking SRO’s initial comments on the report into account, the contractor shall submit the 1-page summary to the COR for SRO review. The contractor shall provide web-ready (i.e. 508-compliant) versions of the report and shorter summaries to the COR (final report) within two weeks after the report is cleared by SRO.

9.3 Snapshot. The contractor shall submit an outline for a study snapshot. The study snapshot shall present data collected as part of an evaluation -- that is policy relevant, important for a Department program office, or has other reason to be included in a standalone product -- but was otherwise not included in an evaluation report. The contractor shall provide a very brief introduction about why and how the snapshot was prepared but the figures, tables, and infographics included shall be the focus of the product with minimal accompanying text. The COR will provide feedback within two weeks.

Once the COR has approved the outline, the contractor shall submit a draft snapshot to the COR on a date specified in the task order. The COR will review the draft snapshot within two weeks. The contractor shall have two weeks to make revisions and submit a revised snapshot, ready for IES’ peer review process overseen by the Standards and Review Office (SRO). The COR will submit the revised snapshot to SRO for review. The COR will provide comments from SRO to the contractor within approximately two months. The contractor shall be responsive to SRO’s comments and revise the snapshot until it clears SRO. Taking SRO’s initial comments on the snapshot into account, the contractor shall submit the snapshot to the COR for SRO review. The contractor shall provide web-ready (i.e. 508-compliant) versions of the snapshot to the COR (final snapshot) within two weeks after the report is cleared by SRO.

9.4 Supplementary Analyses

The contractor shall submit an outline for an interim or post-study-completion analyses that will address issues of interest to researchers or practitioners that are beyond the study’s main research questions. The COR will provide feedback within two weeks.

Once the COR has approved the outline, the contractor shall submit draft supplementary analyses to the COR on a date specified in the task order. The COR will review the draft analyses within two weeks. The contractor shall have two weeks to make revisions and submit revised analyses, ready for IES’ peer review process overseen by the Standards and Review Office (SRO). The COR will submit the revised analyses to SRO for review. The COR will provide comments from SRO to the contractor within approximately two months. The contractor shall be responsive to SRO’s comments and revise the analyses until they clear SRO. Taking SRO’s initial comments on the analyses into account, the contractor shall submit them to the COR for SRO review. The contractor shall provide web-ready (i.e. 508-compliant) versions of the analyses to the COR (final supplementary analyses ) within two weeks after they are cleared by SRO.

9.5 Practitioner Brief. The contractor shall draft a product intended to help educators understand and/or replicate the strategies a parent engagement study found to be promising, effective, or widely used. A practitioner brief might be a full “how-to” guide or pieces of one, such as a description of the core components, the key implementation challenges, or approaches to mitigate those challenges.

The contractor shall submit a draft practitioner brief to the COR on a date specified in the task order. The COR will review the draft practitioner brief within two weeks. The contractor shall have two weeks to make revisions and submit a revised practitioner brief, ready for IES’ peer review process overseen by the Standards and Review Office (SRO). The COR will submit the revised practitioner brief to SRO for review. The COR will provide comments from SRO to the contractor within approximately two months. The contractor shall be responsive to SRO’s comments and revise the practitioner brief until it clears SRO. Taking SRO’s initial comments on the report into account, the contractor shall submit the brief to the COR for SRO review. The contractor shall provide web-ready (i.e. 508-compliant) versions of the practitioner brief to the COR (final practitioner brief ) within two weeks after the report is cleared by SRO.

9.6 Internal Study Memo. The contractor shall conduct data analyses as outlined in an approved plan developed under task 9.1. The contractor shall prepare a memo that includes mostly exhibits (figures, graphs, tables) and bullet points of key findings. The text of the memo shall be no more than 15 pages long but the set of exhibits the text draws upon may be significantly longer.

The contractor shall submit a draft memo for COR review on a date specified in a task order. The COR will review the draft memo within one week. Within two weeks of receiving COR comments, the contractor shall make revisions and submit a final version of the memo.

9.7 Restricted Use File/Data Management Plan. The contractor shall prepare restricted-use data files that contain all quantitative data collected for a study report, as specified by the COR, with all direct personal identifiers removed. The contractor shall provide documentation for the files including variable labels, category labels, the unit of observation, weights, and how missing data were addressed. In the case of qualitative data, the contractor shall prepare restricted-use data files that contain all unabbreviated site visit reports (case studies) with all direct personal identifiers removed.

The contractor shall submit a data management plan in lieu of a restricted use file as required under the IES Public Access Policy for any effectiveness study for which a restricted use file is either not obtainable or inappropriate. The plan shall specify what data sources and samples were used; describe how variables were created; and how data were analyzed.

The contractor shall also provide IES with a risk disclosure analysis for submission to NCES’ Disclosure Review Board (DRB). The contractor shall provide the risk disclosure analysis. Within two weeks of DRB approval of the codebooks and risk disclosure analysis, the contractor shall submit to the COR the restricted-use data files or data management plan consistent with NCES standards[footnoteRef:5]. ASCII or SAS file formats are acceptable. [5: http://nces.the Department .gov/statprog/style.asp]

Revised Study Report Final Study Report Draft Snapshot Revised Snapshot Final Snapshot Draft Supplementary Analyses Revised Supplementary Analyses Final Supplementary Analyses Draft Practitioner Brief Revised Practitioner Brief Final Practitioner Brief Draft Internal Study Memo Final Internal Study Memo Risk disclosure analysis Restricted use data files/Data Management Plan

Task 10--Monthly Communication with Department and Reporting

10.1 Ongoing Communication. The contractor shall provide updates on task order activity and identify issues or problems primarily through email. The contractor shall send invites to participate in initial biweekly and subsequent monthly conference calls with the COR as needed during periods of task order activity. These calls will be used to update the COR on the progress of task order activities and to discuss any issues or anticipated problems and proposed solutions to those problems. Although the contractor may be required to perform this subtask for different task orders, Department expects that the emails and monthly calls will pertain to all current task orders in the aggregate and not that there will be separate monthly calls for each task order. Within two weeks of each call, the contractor shall send a short email summarizing decision made in collaboration with the COR.

10.2 Monthly Reporting. The contractor shall submit one copy of the following reports each month to the COR and CO:

· Monthly Progress Report/Exception Reports. The contractor shall submit monthly progress reports due within 10 workdays after the end of each month. The reports shall summarize the major activities and accomplishments for the reporting period. In addition, they shall provide information for each project task regarding significant findings and events, problems encountered, and staff use. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions which may be needed from the Department. If there are no exceptions, the reports shall state that there are no exceptions. If there are exceptions to the management plan, the contractor shall describe the plan for resolving the problems.

· Monthly Manpower/Expenditure Reports. The contractor shall submit monthly expenditure reports due within 10 workdays after the end of each month. These reports, prepared and signed by the project director, shall summarize the actual personnel assignments for the month just completed, showing for each staff member the hours charged by task. The report shall project similar assignment information for the upcoming month. The reports shall also exhibit expenditures, segregating project costs by individual and by task, and specifying for all travel the locations, duration, and personnel for each trip.

· Security Clearance Spreadsheet. Along with the monthly progress and expenditure reports, the contractor shall submit a spreadsheet that lists each staff member assigned to work on the contract, the required security clearance level of their position (see table below), and their progress toward attaining clearance. After contract award, the COR will provide a spreadsheet template for the selected contractor to use for this reporting purpose.

Project Leadership (Project Director, Deputy Project Director, Principal Investigator) Moderate risk

Researcher (Analyst, Programmer, Statistician, Research Assistant) Moderate risk

Site Outreach Staff (Site Recruiter, Site Coordinator, Site Liaison) Moderate risk

Data Collector (Data Coordinator, Test Proctor, Survey Administrator, Site Visitor, Interviewer, Classroom Observer, Data Coder)

Site Outreach Staff (Site Recruiter, Site Coordinator, Site Liaison) No risk

Researcher (Analyst, Programmer, Statistician, Research Assistant) No risk

Deliverables: Monthly emails Monthly progress reports and security clearance spreadsheets

D. PUBLIC ACCESS, INFORMATION TECHNOLOGY, AND INFORMATION SECURITY REQUIREMENTS

All publications produced under this contract are subject to the IES Public Access Policy[footnoteRef:6][1]. Causal studies are required to have a Data Management Plan, an extension of an Analysis Plan that describes the data used and the study team’s plans for making the data accessible to others (see attachment). Any peer-reviewed work that is not published by IES, such as journal articles or publications on a contractor website, are subject to the IES Public Access Policy for Publications. This requires that the final, peer-reviewed manuscript be submitted to the Educational Resources Information Center (ERIC) and that contractors retain the right for ERIC to display the full text within 12 months of publication. [6: [1] https://ies.the Department .gov/funding/researchaccess.asp ]

The contractor shall ensure that all data in a computerized form shall be suitable for data editing and corrective actions consistent with the U.S. Department of Education’s Information Quality Guidelines (http://www.ed.gov/offices/OCIO/info_quality/final_guide.html) in order to provide for quality control of data entry and yielding verified and accurate records.

The contractor shall ensure that all reports comply with the appropriate technical standards for compliance with Section 508 of the Rehabilitation Act of 1973 as amended, at 36 CFR 1994, and be subject to conformance testing by the Department’s Assistive Technology Program. All Section 508 violations uncovered during the testing process shall be remediated by the vendor at no charge to the Department.

The Department has determined that the work under this contract does not…