Amendment_0001_to_ED-IES-16-R-0014_(SF30).pdf

NSN 7540-01-152-8070 STANDARD FORM 30. (Rev. 10-83) Previous Edition unusable Prescribed by GSA FAR (48 CFR) 53.243

Page of Pages AMENDMENT OF SOLICITATION/MODIFICATION OF CONTRACT 1. Contract ID Code

2. AMENDMENT MODIFICATION NO. 3. EFFECTIVE DATE 4. REQUISITION/PURCHASE REQ. NO. 5. PROJECT NO. (if applicable)

6. ISSUED BY CODE 7. ADMINISTERED BY (If other than item 6) CODE

(x) 9A. AMENDMENT OF SOLICITATION NO.

9B. DATED (SEE ITEM 11)

10A. MODIFICATION OF CONTRACT/ORDER NO.

8. NAME AND ADDRESS OF CONTRACTOR (NO., Street, Country, State and ZIP Code)

CODE FACILITY CODE

10B. DATED (SEE ITEM 13)

11. THIS ITEM ONLY APPLIES TO AMENDMENTS OF SOLICITATIONS

The above numbered solicitation is amended as set forth in item 14. The hour and date specified for receipt of offers is extended, is not extended.

Offers must acknowledge receipt of this amendment prior to the hour and date specified in the solicitation or as amended, by one of the following methods:

(a) By completing items 8 and 15, and returning ______ copies of amendment; (b) By acknowledging receipt of this amendment on each copy of the offer submitted;

or (c) By separate letter or telegram which includes a reference to the solicitation and amendment numbers. FAILURE OF YOUR ACKNOWLEDGMENT TO BE RECEIVED AT THE PLACE DESIGNATED FOR THE RECEIPT OR OFFERS PRIOR TO THE HOUR AND DATE SPECIFIED MAY RESULT IN REJECTION OF YOUR OFFER. If by virtue of this amendment your desire to change an offer already submitted, such change may be made by telegram or letter, provided each telegram or letter makes reference to the solicitation and this amendment, and is received prior to the opening hour and date specified.

12. ACCOUNTING AND APPROPRIATION DATA (If required)

13. THIS ITEM ONLY APPLIES TO MODIFICATION OF CONTRACTS/ORDERS. IT MODIFIES THE CONTRACT/ORDER NO. AS DESCRIBED IN ITEM 14.

Check One

A. THIS CHANGE ORDER IS ISSUED PURSUANT TO: (Specify authority) THE CHANGES SET FORTH IN ITEM 14 ARE MADE IN THE

CONTRACT ORDER NO. IN ITEM 10A.

B. THE ABOVE NUMBERED CONTRACT/ORDER IS MODIFIED TO REFLECT THE ADMINISTRATIVE CHANGES (such as changes in paying office, appropriation date, etc) SET FORTH IN ITEM 14, PURSUANT TO THE AUTHORITY OF FAR 43.103(b).

C. THIS SUPPLEMENTAL AGREEMENT IS ENTERED INTO PURSUANT TO AUTHORITY OF:

D. OTHER (Specify type of modification and authority)

E. IMPORTANT: Contractor is not, is required to sign this document and return _______ copies to the issuing office.

14. DESCRIPTION OF AMENDMENT/MODIFICATION (Organized by UCF section headings, including solicitation/contract subject matter where feasible)

Except as provided herein, all terms and conditions of the document referenced in item 9A or 10A, as heretofore changed, remains unchanged and in full force and effect.

15A. NAME AND TITLE OF SIGNER (Type or print)

16A. NAME AND TITLE OF CONTRACTING OFFICER (Type or print)

15B. CONTRACTOR/OFFEROR 16B. UNITED STATES OF AMERICA

(Signature of person authorized to sign)

15C. DATE SIGNED

(Signature of Contracting Officer)

16C. DATE SIGNED

Attachment Page

PAGE 2 OF 44 ED-IES-16-R-0014/0001

Department Responses to Clarification Questions

ED-IES-16-R-0014

Question 1: For Task 2, can IES clarify whether the contractor will prepare an Annual Work Plan in Year 1? If so, can IES clarify what it means for the Work Plan to be “in alignment with other WWC Content Contracts?”

Department Response:

Yes, the contractor will prepare an Annual Work Plan in Year 1. IES shall work with the PEPPER and other content contractors to ensure that there is a transparent, clear, work plan for reviews that will be conducted across all content contracts. Having the work plans across contracts in alignment means that IES and each content contractor will meet to determine whether there is overlap in content areas that include high school and the transition to college and determine how to appropriately divide the work across contracts.

Question 2: Does the word “topic area” in Subtask 3.1 (“Revise the plan described in the proposal for identifying topic areas for review”) refer to WWC Topic Areas (e.g., Developmental Students in Postsecondary Education), or does it refer to topics of interventions/practices that will be the subject of WWC reviews?

Department Response:

“Topic area” refers to WWC Topic Areas related to postsecondary education and will include existing topic areas and new ones identified by the contractor. The contractor is expected to both refine the current topic areas (as listed on the website) and define topics of interventions for WWC reviews in task 3.1

Question 3: Do the “Proposed plans” described in Subtasks 3.1 and 3.2 refer to plans that will be proposed to the COR after the award of the PEPPER contract or a plan that is included in the proposal responding to the PEPPER RFP?

Department Response:

The Performance Work Statement has been updated to better clarify the requirements for Task 3.

Question 4: Regarding the technical assistance described in Task 7, can IES clarify the audience for the TA and the anticipated number of audience members for the in-person trainings conducted at ED’s offices?

Department Response:

The audiences for technical assistance may include, but are not limited to, IES staff, other ED staff and other contract staff. These trainings may include, for example, technical assistance to provide postsecondary offices across the Department with training to understand WWC reviews.

The number of participants cannot be determined at this time.

PAGE 3 OF 44 ED-IES-16-R-0014/0001

Department Responses to Clarification Questions

ED-IES-16-R-0014

Question 5: Can IES please clarify the number of Practice Guides that are anticipated to be exercised under Optional Task 12 during the period of performance? Also, should vendors assume that the level of effort for Task 12 is commensurate with recent contracts for Practice Guides awarded by ED?

Department Response:

The Performance Work Statement has been revised to better clarify the quantities involved. The estimated level of effort (LOE) for one guide in Task 12 (12.1 and 12.2) is roughly 4,400 hours, which is commensurate with recent contracts for Practice Guides, including the Teaching Academic Content and Literacy to English Learners (ED-IES-12-C-0016) and Dropout Prevention (ED-IES-15-C-0054). This estimated LOE is provided for planning purposes only.

Prospective contractors must ensure that the proposed LOE for this task is commensurate with the technical approach.

Question 6: The description of Optional Task 13 suggests that there is one STAT team across WWC contracts – “the contractor shall contribute to the Statistical, Technical, and Analysis Team (STAT).” Is it the case that the STAT teams will be consolidated across WWC contracts or will each contract have its own STAT team?

Department Response:

The “WWC Investment’s STAT” refers to the collective work of the STAT team or teams funded across all WWC contracts. If Optional Task 13 is exercised, the contractor shall maintain its own distinct STAT team to contribute to the work of the WWC Investment’s STAT, coordinating topic selection with the COR and each other WWC contract STAT team (if applicable) to avoid duplication of effort. For the sake of budgeting for this task, each offeror should assume that its own STAT team will meet via conference call for 90 minutes each month (not counting time to prepare or review background materials), and one contractor staff member will join in a corresponding monthly meeting of a STAT team supported under a separate WWC contract.

Question 7: Are the meetings under subtask 1.2 intended to be biweekly or weekly? The subtask name suggests biweekly but the third sentence of the description references weekly calls between the PD and COR.

Department Response:

The meetings are intended to be bi-weekly.

Question 8: Under Optional Task 12, the cost of a practice guide is significantly dependent on the number of studies identified for review. Is it possible to either (1) change the contract type for Optional Task 12 to cost reimbursable, or (2) include the study reviews in support of the practice guide(s) as a separate, cost-reimbursable subtask? If not, can you provide guidance on the number of reviews we should assume, per guide?

Department Response:

The practice guide will remain firm fixed price and the contractor will make its estimate for the review of studies based on the topic/practices proposed. Vendors may use the level of effort for

PAGE 4 OF 44 ED-IES-16-R-0014/0001

Department Responses to Clarification Questions

ED-IES-16-R-0014

the Teaching Academic Content and Literacy to Elementary and Middle School English Learners Practice Guide—roughly 4,400 hours over two years --as a model for the intended scope. This estimated LOE is provided for planning purposes only. Prospective contractors must ensure that the proposed LOE for this task is commensurate with the technical approach.

Question 9: To whom should we submit the proposal? The RFP specifies that we should submit questions to Stephen Madsen with a cc: to Sabrina Phillips. Should we assume it is the same for the proposal?

Department Response: Yes. The Instructions to Offerors have been revised to better clarify this point.

Question 10: The solicitation covers compliance with the ED IT Security Policy (pp. 12- 19). Are contractors expected to include information about how the contractor will respond to these requirements, or is this section meant to detail what requirements the contractor will need to meet post-award?

Department Response:

The ED IT Security Policy section details the requirements the contractor that the contractor will need to meet post-award.

Question 11: On p. 5 of 13 from Instructions to Offerors, the section on Value Added Suggestions states that “offerors are encouraged to suggest technical improvements to the work.”

To what area of work does this refer? Does this refer to Tasks 1 through 13 in the Scope of Work

(pp. 5-11 of the Performance Work Statement) or more specifically to Optional Task 13, or to something else?

Department Response:

Contractors are encouraged to suggest technical improvements to all aspects of the WWC work.

For example, the contractor might identify a new approach for developing supporting materials related to Practice Guides. They could provide ideas to take advantage of alternative communication strategies related to WWC products. Or they might offer suggestions for improvements in the algorithm to identify which studies receive “significant media attention” for Quick Review products.

Question 12: On p. 7 of 23 of the Performance Work Statement, the document states that the “contractor shall apply appropriate standards and procedures provided by ED and an organizational firewall to avoid any conflict of interest problems…” What type of processes and procedures are recommended by IES in the creation of “an organizational firewall”? Are there specific guidelines? Should this be included in the Appendix to the business proposal as part of the Conflict of Interest Plan and Certificate?

Department Response:

The contractor is expected to include in its technical proposal an approach that includes specific processes and procedures to create an organizational firewall that will prevent conflicts of interest. For example, vendors may propose a process that ensures that study authors are not

PAGE 5 OF 44 ED-IES-16-R-0014/0001

Department Responses to Clarification Questions

ED-IES-16-R-0014

reviewing their own studies, subcontracting the review of such studies to other reviewers.

Vendors may also require reviewers to certify that they do not have a conflict of interest with regards to a particular study before reviewing it.

Question 13: On page 78, the solicitation states that "Offerors shall provide evidence of prior successful evaluation work in areas including large-scale educational evaluations that involve:

challenging large-scale study recruitment; random assignment in an education setting;

implementation support; collection and analysis of quantitative and qualitative data including sophisticated correlational analyses and collection/analyses of classroom observation data;

content knowledge of postsecondary education and preparation interventions; prior successful work in conducting studies of a similar nature." Should past performance projects focus more on evaluation studies conducted or on initiatives similar to that of the WWC-PEPPER contract?

Department Response:

The past performance volume should address initiatives similar to that of the PEPPER contract, and reflect an understanding of effectiveness research and the ability to carry out effectiveness studies. IES expects this combination of Past Performance to demonstrate a strong ability to be able to review research studies, provide practical suggestions for WWC standards and improvement, and to provide IES with a contractor that understands all aspects of high quality studies that will impact reviews.

PAGE 6 OF 44 ED-IES-16-R-0014/0001

Contract Value & Period of Performance Version Date:

Period of Perf. Fixed Price CR Fee on CR Total Fixed Price CR Fee on CR Total Base Period Baseline Tasks 9/30/16 - 9/29/17 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Base Period Optional Tasks 9/30/16 - 9/29/17 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Opt. Period 1 Baseline Tasks 9/30/17 - 9/29/18 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Opt. Period 1 Optional Tasks 9/30/17 - 9/29/18 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Opt. Period 2 Baseline Tasks 9/30/18 - 9/29/19 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Opt. Period 2 Optional Tasks 9/30/18 - 9/29/19 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Opt. Period 3 Baseline Tasks 9/30/19 - 9/29/20 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Opt. Period 3 Optional Tasks 9/30/19 - 9/29/20 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Opt. Period 4 Baseline Tasks 9/30/20 - 9/29/21 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Opt. Period 4 Optional Tasks 9/30/20 - 9/29/21 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00

Total: $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 $0.00 Total: $0.00 $0.00 $0.00 $0.00

Total Value (including unexercised options) Funded Value (including unexercised options)

$0.00 $0.00

This is a "Cost Plus Fixed Fee" contract. In the chart above, "Fixed Price" refers to the Firm Fixed Price work, "CR" refers to the Cost Reimbursement work (not including fixed fee), and "Fee on CR" refers to the fixed fee to be paid on CR work proposed.

PAGE 7 OF 44 ED-IES-16-R-0014/0001

Line Items: ED-IES-16-R-0014 Fixed Fee on CR:

Version Date:

Line Item # Line Item Name Amount Base Period Baseline Tasks (Firm Fixed Price) Base Period Baseline Tasks (Cost Reimbursement, excluding Fee on CR) Total Basline:

Base Period Baseline Tasks (Fee on CR) $0.00 $0.00 Base Period Optional Task 11 (Firm Fixed Price) Total Options:

Base Period Optional Task 12 (Firm Fixed Price) (total for 3 units of Task 12) $0.00 Base Period Optional Task 13 (Cost Reimbursement, excluding Fee on CR) Total:

Base Period Optional Task 13 (Fee on CR) $0.00 $0.00 Option Period 1 Baseline Tasks (Firm Fixed Price) Option Period 1 Baseline Tasks (Cost Reimbursement, excluding Fee on CR) Total Basline:

Option Period 1 Baseline Tasks (Fee on CR) $0.00 $0.00 Option Period 1 Optional Task 11 (Firm Fixed Price) Total Options:

Option Period 1 Optional Task 12 (Firm Fixed Price) (total for 3 units of Task 12) $0.00 Option Period 1 Optional Task 13 (Cost Reimbursement, excluding Fee on CR) Total:

Option Period 1 Optional Task 13 (Fee on CR) $0.00 $0.00 Option Period 2 Baseline Tasks (Firm Fixed Price) Option Period 2 Baseline Tasks (Cost Reimbursement, excluding Fee on CR) Total Basline:

Option Period 2 Baseline Tasks (Fee on CR) $0.00 $0.00 Option Period 2 Optional Task 11 (Firm Fixed Price) Total Options:

Option Period 2 Optional Task 12 (Firm Fixed Price) (total for 3 units of Task 12) $0.00 Option Period 2 Optional Task 13 (Cost Reimbursement, excluding Fee on CR) Total:

Option Period 2 Optional Task 13 (Fee on CR) $0.00 $0.00 Option Period 3 Baseline Tasks (Firm Fixed Price) Option Period 3 Baseline Tasks (Cost Reimbursement, excluding Fee on CR) Total Basline:

Option Period 3 Baseline Tasks (Fee on CR) $0.00 $0.00 Option Period 3 Optional Task 11 (Firm Fixed Price) Total Options:

Option Period 3 Optional Task 12 (Firm Fixed Price) (total for 3 units of Task 12) $0.00 Option Period 3 Optional Task 13 (Cost Reimbursement, excluding Fee on CR) Total:

Option Period 3 Optional Task 13 (Fee on CR) $0.00 $0.00 Option Period 4 Baseline Tasks (Firm Fixed Price) Option Period 4 Baseline Tasks (Cost Reimbursement, excluding Fee on CR) Total Basline:

Option Period 4 Baseline Tasks (Fee on CR) $0.00 $0.00 Option Period 4 Optional Task 11 (Firm Fixed Price) Total Options:

Option Period 4 Optional Task 12 (Firm Fixed Price) (total for 3 units of Task 12) $0.00 Option Period 4 Optional Task 13 (Cost Reimbursement, excluding Fee on CR) Total:

Option Period 4 Optional Task 13 (Fee on CR) $0.00 $0.00

Baseline Tasks FFP: $0.00 Baseline Tasks CR: $0.00

Baseline Tasks Fee on CR: $0.00

TOTAL BASELINE TASKS: $0.00

Optional Tasks FFP: $0.00 Optional Tasks CR: $0.00

Optional Tasks Fee on CR: $0.00

TOTAL OPTIONAL TASKS: $0.00

TOTAL CONTRACT VALUE: $0.00

PAGE 8 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

ED-IES-16-R-0014

U.S. Department of Education Institute of Education Sciences

National Center for Education Evaluation and Regional Assistance

Performance Work Statement for the contract titled

WHAT WORKS CLEARINGHOUSE:

POSTSECONDARY EDUCATION AND POSTSECONDARY PREPARATION

EVIDENCE REPORTING (WWC-PEPPER)

Introduction

The U.S. Department of Education, via the National Center for Education Evaluation and Regional Assistance (NCEE) of the Institute of Education Sciences (IES), has the need for a What Works Clearinghouse (WWC) contractor that will lead reviews in the areas of postsecondary education, the transition to college, and programs that take place in middle and high school with the long-term goal of impacting postsecondary attainment. The Postsecondary Education and Postsecondary Preparation Evidence Reporting (PEPPER) contract will continue the work started through the previous Postsecondary contract for the WWC (Evidence Review and Support, ED-IES-12-C-0084). Within the postsecondary and related content areas described above, this contract will primarily conduct comprehensive reviews for WWC intervention reports, develop practice guides, produce products based on quick reviews and individual study reviews, and create new products to disseminate the findings of the WWC. The period of performance for this contract is five years and includes optional tasks that can be exercised at the discretion of IES, pending available funds.

Authorizing Legislation Section 171(b) of the Education Sciences Reform Act of 2002 states that the mission of the National Center for Education Evaluation and Regional Assistance in the Institute of Education Sciences shall be to provide technical assistance, and to encourage the use of scientifically valid education research and evaluation throughout the United States. The WWC encourages the use of scientifically valid education research by serving as the public’s central and trusted source on what works in education, providing accessible information on study quality.

Background

The WWC was established in 2002 by the Department/IES to provide educators, policymakers, researchers, and the public with a transparent way to determine whether there is evidence of effectiveness behind a program or practice and the extent to which those findings can be trusted.

The WWC was charged to develop and establish WWC Study Evidence Standards and a systematic review system. The goal was to promote better informed education decision-making

PAGE 9 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

ED-IES-16-R-0014

through a web-based dissemination system that features comprehensive, systematic, high-quality reviews1 of studies on the effectiveness of educational interventions (i.e., programs, products, practices, and policies). The website, whatworks.ed.gov is the primary “face” of the WWC, displaying the results of the systematic review process.

Nearly 15 years after the WWC was first authorized, the mission for promoting informed decision-making has continued. At this time, the WWC is focused on providing timely, comprehensive reviews of interventions, establishing and refining state of the art evidence standards for the field of education, providing data to a wide-range of stakeholders as part of the government’s Open Data Initiative, and disseminating information to the public.

Current WWC Contract Mechanisms The WWC investment currently operates through five distinct contract vehicles:

ED-IES-13-C-0010: What Works Clearinghouse Content ED-IES-13-C-0013: What Works Clearinghouse Website ED-IES-12-C-0084: Evidence Review and Support for the What Works Clearinghouse

(herein referred to as the WWC Postsecondary Contract) ED-IES-15-D-0003: What Works Clearinghouse Reviews, Reporting, Dissemination, &

Development ED-IES-15-C-0054: What Works Clearinghouse Dropout Prevention Practice Guide

It is expected that these contracts work in a coordinated fashion (at the direction of IES as specified later in this PWS) to support the work of the entire WWC investment. The “WWC Pre-K to 12 Content Contract” (ED-IES-13-C-0010) focuses on reviews covering pre-kindergarten through 12th grade. The current “WWC Evidence Review and Support Contract” (ED-IES-12-C-0084) focuses on reviews in the area of postsecondary access and success. Tasks in both contracts focus on generating comprehensive reviews of existing effectiveness studies.

These two contractors each maintain a WWC Statistical and Technical Analysis Team (STAT), produce key products (listed below), and manage inquiries that arise in response to reviews. It is imperative that the new postsecondary contractor (PEPPER) work in a coordinated manner with the “WWC Pre-K to 12 Content” contractor in order for IES to carry out an investment-wide plan for prioritizing topic areas and studies for review; creating and updating review protocols;

conducting communication and dissemination activities; and other cross-investment work. The details of this coordinated work are described in Task 2.

The “WWC Website Contract” (ED-IES-13-C-0013) supports the production and implementation of all WWC materials posted on the WWC website as well as the infrastructure of the WWC website itself. IES expects that the PEPPER contractor will work in coordination with the website contractor by submitting materials that will be posted by the web contractor, using established procedures.

1 A systematic review is a review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review. For further information about the WWC Systematic Review Process, refer to www.whatworks.ed.gov/reviewprocess/review.html.

PAGE 10 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

ED-IES-16-R-0014

The purpose of the Reviews, Reporting, Dissemination, and Development Contract (R2D2) (ED- IES-15-D-0003) is to conduct timely reviews of studies submitted as part of the Department’s evidence-based grant competitions. In addition, there are tasks in this contract for creating and updating Intervention reports; creating Single Study Reviews for a subset of studies reviewed for grant competitions; and identifying innovative ways to further develop the WWC as a recognizable brand to the public.

Finally, the purpose of the WWC Dropout Prevention Practice Guide Contract (ED-IES-15-C- 0054) is to work in consultation with an expert panel to develop and publish an IES practice guide with recommendations for practitioners about how to prevent students from dropping out of high school. The PEPPER contractor will need to coordinate with the Dropout Prevention contractor to ensure alignment on any Dropout Prevention protocols and to understand the status of dropout prevention studies that may be considered for work under the PEPPER contract.

All WWC contracts must continue to work in a cohesive, coordinated manner to represent the entire WWC investment, at the direction of IES.

Current WWC Products Each of the WWC contracts is responsible for creating or featuring products that translate study reviews into findings that are accessible to the public. To develop these products, the WWC uses objective and transparent standards and procedures to assess the scientific merit of studies of the effectiveness of education interventions. Underlying each of these reviews is a Study Review Guide (SRG), which is completed by WWC certified reviewers to assess the quality of the research according to established WWC standards (access the current SRG template here:

http://ies.ed.gov/ncee/wwc/StudyReviewGuide.aspx). Below are the central WWC products:

Intervention Reports. Intervention reports are a summary of all studies published during a specific time period that examine the effectiveness of an intervention. Using its objective and transparent standards, the WWC rates the quality of the evidence in each study. For studies that meet WWC evidence standards (with or without reservations), the WWC combines the findings to generate overall estimates of the size of effects for the intervention.

The WWC also provides a rating of the level of evidence on the intervention’s effectiveness, taking into consideration the number of studies, the sample sizes, and the magnitude and statistical significance of the estimates of effectiveness.

Practice Guides. Practice guides contain practical recommendations that educators can use to address specific challenges in their classrooms and schools. The recommendations are based on reviews of research as well as the expertise and professional judgments of a panel of nationally recognized experts that includes both researchers and educators.

Reports on Individual Studies. The WWC conducts reviews of individual studies that are published as Single Study Reviews or are added without reports to the WWC reviewed database http://ies.ed.gov/ncee/wwc/ReviewedStudies.aspx.

PAGE 11 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

ED-IES-16-R-0014

More details on the current products can be found in Appendix A. IES continues to be interested in creating new product lines for the WWC investment that are: consistent with the mission of the WWC; creative and engaging; and provide systematic information on a topic or content area that are understudied or lack rigorous research.

Prior to 2012, IES directed the WWC investment to review interventions and practices that directly affect K-12 educational outcomes. Since the award of the first postsecondary WWC contract in 2012, nearly 300 studies related to postsecondary education research have been added to the Database of Reviewed Studies. To date new systematic review topic areas have been opened, such as the Transition to College and Developmental Education, and new review protocols for postsecondary education research studies have been developed that lay the foundation for future reviews in this area. The PEPPER contractor will be responsible for continuing the review of studies in the postsecondary area.

Purpose

The main purpose of this contract is to conduct reviews and develop products that will continue to expand the content of the topic areas within postsecondary education review area, including the transition to college. Student achievement will remain the primary outcome focus of the WWC; however the WWC also seeks to review studies that examine other relevant education outcomes for students, such as persistence, attainment of credentials (certificates and degrees), mental health and anxiety as they approach and progress through their education, and labor market outcomes.

A secondary goal of this contract is to introduce new products that enhance and expand the content and reach of the WWC while also reflecting a commitment to the rigorous research standards that are the foundation of the WWC. Therefore, new products developed under this award shall be clearly applicable and useful to a variety of relevant audiences (e.g., postsecondary practitioners, decision-makers, and researchers). Ideas are invited for new products that improve and expand the content, reach, and usefulness of the WWC while also maintaining or enhancing the WWC commitment to rigorous standards and high quality research. Developing new products that rely on the systematic review process and represent a defined, transparent procedure has been a challenge. In this contract, it is expected that offerors will propose innovative new product lines and consider proposing a team with experience developing and disseminating creative work. For the duration of this contract, these new products shall be limited to content area covered by this contract. Other topic areas may be proposed if the optional tasks are exercised.

Period of Performance

This contract contains a 12-month base period and four 12-month option periods, with periods of performance as follows:

Base Period (September 30, 2016-September 29, 2017) Option Period 1 (September 30, 2017-September 29, 2018) Option Period 2 (September 30, 2018-September 29, 2019) Option Period 3 (September 30, 2019- September 29, 2020)

PAGE 12 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

ED-IES-16-R-0014

Option Period 4 (September 30, 2020- September 29, 2021) Scope of Work

This section specifies the tasks and subtasks that the contractor shall perform. Offerors shall propose interim deliverables and a payment schedule for all tasks and subtasks according to the cost-reimbursement or fixed price designation. All deliverables shall be submitted to ED’s Contracting Officer’s Representative (COR) and ED’s Contracting Specialist (CS) electronically unless otherwise noted.

The COR reserves the right to inspect the contractor’s work while in process. The contractor shall submit to the COR any progress report or other written documentation of work to date that the COR requests to facilitate that inspection, according to contract clauses Federal Acquisition Regulation (FAR) 52.246-4 (fixed price) and 52.246-5 (cost-reimbursement).

This Performance Work Statement (PWS) contains both Firm Fixed Price (hereafter referred to as Fixed Price) tasks/subtasks as well as Cost Plus Fixed Fee (hereafter referred to as Cost Reimbursement) tasks/subtasks. Each type is noted below. Baseline tasks shall be performed in the base period and each exercised option period, unless otherwise noted. Optional tasks, if exercised by the Government, are available in the base period and each option period, unless otherwise noted.

Task 1 Overall Contract Management and Communication

Task 1 describes the overall contract management, communications, and transition for the contract. The contractor shall assume responsibility for ensuring that all contractor staff are in compliance with ED’s contractor security clearance process, in accordance with security-related contract clauses.

Subtask 1.1 Kickoff Meeting (Fixed Price). The contractor’s project director and key project staff (up to three) shall meet with the WWC COR and other appropriate ED staff within two weeks following the award date to discuss details regarding the tasks outlined in the proposal, scheduling activities, and other issues related to the contract. The meeting shall be at ED’s offices in Washington, D.C. The contractor shall provide an agenda for the meeting within five work days after the effective date of the contract. The contractor shall come prepared to identify any areas of concern and to suggest ways of responding to these concerns. Within two weeks following the meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting and how each will be addressed.

Subtask 1.2 Bi-weekly update conference calls (Fixed Price). The contractor’s project director shall submit bi-weekly (i.e., once every two weeks) abbreviated notes to the COR on the status of each of the tasks under the contract, including details on the status of each review underway.

The contractor shall submit the notes at least one hour prior to regularly scheduled bi-weekly telephone updates. Outstanding issues or actions requiring follow up by either the COR or the contractor’s project director will be discussed in bi-weekly conference calls between the contractor’s project director and the COR. The contractor shall update the abbreviated status report based on discussions and submit the report to the COR within one day following the call.

PAGE 13 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

ED-IES-16-R-0014

Subtask 1.3 Periodic meetings (Cost Reimbursement). The project director and up to two additional key staff shall meet with the COR and other appropriate ED staff in Washington, D.C.

on an as needed basis, in person or by conference call, not to exceed four times per year. The purpose of the meeting is to brief ED on the progress of the contract and to discuss issues as they arise. Within two weeks following each meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting, and how each will be addressed.

Subtask 1.4 Monthly Reporting (Fixed Price). In addition to the reports that are required for each task as described, for cost reimbursable tasks, the contractor shall include discussion of progress, issues, and problems in a monthly progress report. The monthly reports shall include a budget summary of costs per task and per subtask, including expenditures in relation to progress on content development and other work streams, so that the COR can easily see how much has been spent on work in a given task/subtask. The reports shall also specify the extent to which the project is on schedule, briefly describe the activities planned for the next month, identify and discuss significant deviations from the substantive and time factors in the management plan, and identify and discuss any decisions that may be needed from ED.

Subtask 1.5 Quality Control (Cost Reimbursement). The contractor shall create procedures for collecting and analyzing data on the quality and accuracy of task order activities delivered and the products developed under this contract. These quality control measures may include tools such as internal quality checks on WWC study reviews to ensure that all reviewers are accurate and up to date on WWC procedures.

Subtask 1.6 Collaboration and Coordination with the Other WWC Contractors (Cost Reimbursement). As the WWC investment continues to grow, it is critical that all contractors operate in a coordinated, collaborative fashion. Key personnel from the PEPPER Contract shall have at least bimonthly communication with the project director of the other WWC contracts (via conference call), as requested by the COR. The content of these phone calls shall include discussion about new developments in the WWC standards, protocol requirements, methodological issues, expected volume of work, issues and challenges, any upcoming changes that may be occurring at the WWC, or other relevant topics. The COR will participate in these phone calls, as needed. The contractor shall include a memo summarizing key issues and concerns raised at the meeting and how each will be addressed as part of the monthly report.

This memo shall be provided two days after the meeting.

Task 2 Develop the Annual Forecasting Work Plan (Cost Reimbursement)

In direct consultation with the COR and IES leadership, the contractor shall create an annual forecast and workplan for reviews conducted under this contract. The contractor shall discuss this workplan with both IES and other WWC contractors, as identified by IES, to ensure that the WWC uses a clear, defensible prioritization for reviewing studies and covers a wide range of non-overlapping content areas on an annual basis. The detailed Annual Work Plan shall include the topics proposed for WWC reviews, the type of reports that will be produced, and specific activities that the contractor will carry out in performance of each task and subtask.

PAGE 14 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

ED-IES-16-R-0014

The contractor shall include all tasks and subtasks in the Annual Work Plan, and in particular shall identify:

1. A prioritization scheme that identifies topics for review based on a clear metric

2. The review process that will be implemented

3. The type of products that the contractor will develop and produce

4. A production schedule with timelines, deliverables, and pricing for each deliverable, and

5. Proposed quality assurance procedures

The contractor shall prepare and submit to the COR semi-annual updates on the Annual Work Plans. The contractor shall apply appropriate standards and procedures provided by ED and an organizational firewall to avoid any conflict of interest problems among the principal investigators leading the reviews, the contractor’s staff and subcontract staff, and other outside experts in performing work under this contract. This plan shall be set by the summer of 2018 (Year 2) in alignment with other WWC Content Contracts.

Task 3 Identify Topics for Review and Establish Topic Area Teams (Cost

Reimbursement)

Task 3 describes the topic areas that will be reviewed, the process by which topic area teams will review studies, and a requirement to summarize across an existing topic area in a user-friendly format. Since the postsecondary area includes a range of interventions, practices, and strategies that are not branded and often more difficult to identify, this task ensures that contractors will identify a broad range of new topics areas that could be covered with this contract.

Subtask 3.1 Revise theCreate a plan described in the proposal for identifying topic areas for review. After receiving feedback from the COR, tThe contractor shall revise thecreate a plan submitted in the proposal for reviewing the literature primarily in the area of postsecondary education and the transition to college. After receiving feedback from the COR, the contractor shall update the plan. The purpose of this scan is to identify the key topic areas that will be reviewed. The contractor shall identify topics within the field of postsecondary education on which effectiveness research studies have been conducted, as well as where there are gaps in the research base on the effectiveness of a program, policy, or practice.

After COR review and approval, the results of this scan shall be submitted in a suitable format for publishing on the WWC website (maintained by the WWC contractor and hosted by IES) to provide information about upcoming reviews. See Appendix B for information about IES policies for web posting.

Subtask 3.2 Revise Create athe proposed plan for topic area leadership structure and process.

After receiving feedback from the COR, tThe contractor shall revise create athe plan presented in the proposal to identify content area experts, methodological experts, WWC certified reviewers, and quality assurance procedures that are aligned with the WWC streamlined review process as described in the WWC Standards and Procedures Handbook. After receiving feedback from the COR, the contractor shall update the plan.

PAGE 15 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

Task 4 Conduct WWC Reviews in a Variety of Topic Areas (Cost Reimbursement)

Task 4 describes the systematic reviews of groups studies in identified topic areas and of single studies (identified as quick reviews, single study reviews, or IES or other ED-funded studies).

Subtask 4.1 Develop and produce Intervention Reports. The contractor shall develop, at a minimum, three WWC Intervention Reports per year that result from the conduct of WWC systematic reviews under Task 3. A list of Intervention Reports and deliverables under this task shall be incorporated into the Annual Plan under Task 2.

Subtask 4.2 Conduct reviews at the request of IES. The contractor shall conduct an annual WWC Review of IES-funded research studies to assess whether study designs, or preliminary or final study results meet WWC Evidence Standards, or whether findings indicate intervention effectiveness. The COR shall identify the studies to be reviewed by the contractor and provide a template for reporting the results. At the close of the review process, the contractor shall provide to the website contractor citations for the studies reviewed so that they can be entered into the WWC reviewed studies database. The contractor shall plan to conduct up to 30 reviews at the request of IES on an annual basis.

Subtask 4.3 Conduct Single Study Reviews. The contractor shall apply WWC standards to review individual studies, as defined by the single study review protocol. Studies that receive notable media mention shall be reviewed first as Quick Review Blasts, followed by a Single Study Review. As part of the Annual work plan in Task 2, the contractor shall propose a method for conducting single study reviews and determine the amount and frequency of these reviews.

For estimation purposes, the contractor shall plan to conduct up to seven Single Study Reviews in each year of the contract. The contractor shall also consider ways to integrate these SSRs into products such as intervention reports and practice guides where possible.

Task 5 Contribute to the WWC’s Ongoing Evaluation and Refinement of WWC

Standards and Procedures (Cost Reimbursement)

The contractor shall conduct an initial analysis of WWC standards at the beginning of the contract and submit a proposed plan for changes and improvements, indicating the highest priority items and plans for addressing those high-priority items during this contract period. In addition, the contractor shall contribute updates to the WWC’s investment-wide procedures at least twice per year. As part of this work, the contractor shall contribute to updates the WWC Handbook of Standards and Procedures.

Task 6 Develop and Produce New Products for Practitioners, Researchers and/or

Policymakers (Cost Reimbursement)

Developing new products for a variety of audiences is the focus of Task 6.

The contractor shall propose, develop and produce new products with COR input and approval.

The contractor shall incorporate the associated costs into the Annual Work Plan in Task 2.

PAGE 16 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

Subtask 6.1 Prospectus for Each New Product . The contractor shall develop a prospectus for each individual new product that specifies the: 1) methods for identifying topic areas and practices within topic areas; 2) protocol development; 3) review process; 4) cost estimate; and 5) timelines and deliverables. The plan shall be incorporated into the Annual Plan under Task 2.

Work will be directed by IES based on the broad WWC Annual Workplan.

Subtask 6.2 Develop and produce a new review product. The new product will review the extant evidence in a related topic area and rate the quality of the evidence and identify where there are gaps in the evidence. The contractor shall develop and carry out a production process leading to the release of the new product on the WWC website, according to the COR approved New Product Work Plan in Task 6.1. The contractor shall revise the plan submitted in the proposal that describes examples of proposed product types. The revised plan should address COR comments and feedback provided to the contractor at the startup meeting with ED no later than 4 weeks after the effective date of the contract. The COR will provide comments within 2 weeks.

The contractor shall incorporate COR comments and submit a final plan for new product development within 10 weeks after the effective date of the contract. The contractor shall incorporate the associated costs into the Annual Work Plan in Task 2. The contractor shall implement the plan for new products upon COR approval.

Task 7 Conduct Technical Assistance Trainings on the WWC Review Process (Cost

Reimbursement)

As determined by the COR, the contractor shall provide technical assistance on the WWC Review Process at an Department of Education location in Washington, DC no more than two times per year. The contractor shall also conduct up to two technical assistance trainings via webinar each year. Within 1 week of receiving the COR request for training, the contractor shall develop a timeline to deliver the training and training materials. The COR will comment on the timelines and training materials within 2 weeks of receipt. The contractor shall deliver final training materials within 2 weeks of COR feedback.

Task 8 Develop and Implement a Communications Plan that Contributes to the

WWC Investment’s Overall Communication Plan (Cost Reimbursement)

Task 8 describes the activities associated with developing and implementing a communications plan that will contribute to the investment-wide WWC communications plan.

Subtask 8.1 Develop communications plan. The contractor shall develop a communications plan and incorporate it into the Annual Work Plan. The plan should contribute to the overall WWC-investment communications plan. The contractor shall propose a communications plan with ideas that improve and expand the content, reach, and usefulness of the WWC PEPPER contract. The contractor shall propose new supplemental products that feature the WWC review process and products in a clear, accessible manner. These products may include but are not limited to, new reports or summaries, infographics, videos, webinars, podcasts, guides, or activities, including those carried out on social media. These products will be developed in consultation with the WWC Website contractor to ensure they be posted on the website.

PAGE 17 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

Subtask 8.2 Implement communications plan. The contractor shall implement the communications plan upon COR approval. The communications plan may include travel to conferences to present/disseminate WWC findings with COR approval.

Task 9 Conflict of Interest Policy (Fixed Price)

The contractor shall develop and maintain a conflict of interest policy to assure WWC work and associated personnel are separated from other contractor(s) interests, including reviews that include studies conducted by contractors and personnel or other research syntheses and dissemination projects. The draft of this policy shall be submitted 2 weeks after the kickoff meeting. The COR shall review the plan within 2 weeks. The contractor shall make revisions and submit a revised final policy within 2 weeks of receiving COR comments.

Task 10 Help Desk (Cost Reimbursement)

The WWC Help Desk is responsible for managing all communications that come in through the WWC website, email inbox, or toll-free phone line. All communications are responded to in a timely manner, and a thorough record of all communications must be kept. Statistics are maintained for monthly and annual reporting to IES to include but not limited to the number, types, and topics of inquiries received and responded to by the WWC Help Desk. The contractor shall be responsible for managing the range of inquiries related to the PEPPER contract and all products produced under this area. All Quality Review Team responses will be provided to the COR for review and approval.

Optional Task 11 Transition of Contract Materials (Fixed Price)

At the completion of the final Option Period exercised (or of the Base Period if the option(s) are not exercised), the contractor shall engage in appropriate transition activities to a new contractor.

These shall include:

A written report detailing the key elements and processes of the project that would make a smooth transition possible completed three months before the end of the contract;

Documentation of content meeting the definition of Federal records and a plan for transferring electronic permanent records to the COR

Meetings and/or phone calls, as appropriate, with the new contractor; and One month before the end of the contract, the contractor shall be prepared to inventory and transfer to the COR, or to an ED contractor or other ED agent, all materials, including all files, software, architecture, programming, codes, analysis tools, databases, forms, reports, search engine features, and all other features and supporting documentation that were prepared or purchased by the contractor under the contract. The contractor shall provide all software, databases, interfaces, and data under a no-cost, no-strings license to another contractor, the COR, or an ED agent in case this contract is terminated. The contractor shall provide training to ED staff and others designated by the COR to facilitate a smooth transfer of operations so that ED is able to manage the operation of the database, website, and search engine after this contract ends.

PAGE 18 OF 44 ED-IES-16-R-0014/0001

Performance Work Statement Version Date: AprilJuly 19, 2016

Optional Task 12 Develop, Produce, and Update WWC Practice Guides (Fixed Price)

Optional Task 12 may be exercised by the Government as many as three time per contract period (i.e., three times in the base period, three times in Option Period 1, three times in Option Period 2, etc.)

Subtask 12.1 Create a Practice Guide work plan. The contractor shall revise the draft Practice Guide work plan submitted in the proposal that specifies the: 1) methods for identifying practice guide topic areas and practices within topic areas; 2) protocol development; 3) review process, including designation of members of…