REL Peer Review PWS - 7-6-22.docx

PERFORMANCE WORK STATEMENT

PEER REVIEW FOR REGIONAL EDUCATIONAL LABORATORIES

I. Introduction

The National Center for Education Evaluation and Regional Assistance (NCEE) is required to provide a system for technical and peer review of applied research activities, research-based reports, and products produced by the ten Regional Educational Laboratories (RELs). This contract will include the refinement of existing review criteria, thorough review of REL applied research and development proposals and products, and editing and production of REL products.

II. Background

The REL Program is administered by NCEE at the Institute of Education Sciences (IES) in the US Department of Education. The purpose of each of the ten REL contracts is to assist practitioners and policymakers in their work to improve outcomes for learners in its region by supporting stakeholders in the generation and use of research, evidence, and evidence-based practices. To achieve that purpose, RELs: (1) conduct applied research and development; (2) design and implement training, coaching, and technical support activities that emphasize building capacity to use data and information to drive change; and (3) disseminate scientifically valid research, evidence-based practices, and supporting materials that allow stakeholders to apply this knowledge to their own practice.

All REL applied research and development products are required to undergo rigorous external peer review. Peer review is a critical component of the quality assurance process at IES. This ensures that all applied research and development products meet the IES standards for scientifically valid research before being published. In this way, policymakers and practitioners, the primary users of REL applied research and development products, can be assured that these products have met high standards for scientific quality and that the information is valid, reliable, and can be trusted. The REL Peer review process involves review by content and methodology experts who have no conflicts of interest related to the product being reviewed. The peer review focuses on relevance and significance of the product, the data and analysis methods used, and the presentation of the goals, data and methods, and findings and implications.

This contract will support the REL program in achieving its goals by providing independent peer review of REL research proposals and products.

III. Legislative Authority

The current authorization for the Regional Educational Laboratories program is under the Education Sciences Reform Act (ESRA) of 2002, Part D, Section 174, (20 U.S.C. 9564), administered by the Institute of Education Sciences’ (IES) National Center for Education Evaluation and Regional Assistance (NCEE). Under Section 174(5) Standards, the legislation authorizes the establishment of a process for technical quality and peer review for NCEE products and those produced by the RELs:

The Evaluation and Regional Assistance Commissioner shall establish a system for technical and peer review to ensure that applied research activities, research-based reports, and products of the regional educational laboratories are consistent with the research standards described in section 134 and the evaluation standards adhered to pursuant to Section 173(a)(2)(A)(B).

IV. Contract Type

The contract type is Cost Plus Fixed Fee (CPFF).

V. Period of Performance

The period of performance for this contract is 60 months and consists of a Base Period and three Option Years whose durations are as follows:

· Base Period: 24 months

· Option Year 1: 12 months

· Option Year 2: 12 months

· Option Year 3: 12 months

VI. Scope of Work

The purpose of this contract is to review REL research proposals and products to assure alignment with NCEE standards and goals. The specific objectives are as follows:

· Refine and improve the review criteria and procedures as needed.

· Review proposals and products for REL research projects, tools, toolkits, and new REL-developed interventions.

· Provide as-needed guidance for the improvement of REL products.

· Update the REL Writers and Style Guide and REL report templates as needed.

· Provide editing and production of REL products.

· Present webinars for REL staff as needed.

· Conduct as-needed analytic and methods oriented projects (optional task)

Additional information on REL applied research and development proposals and products that will go through the peer review and production process includes:

· REL research reports are typically published as a report of up to 15 pages, a separate appendix file, and either a one-page or a four-page summary. RELs may also propose an alternative product format, such as a PowerPoint slide deck or infographic; alternative products such as these will be peer reviewed but will not require editing or production by the contractor.

· RELs may develop analytic tools intended to support stakeholders’ capacities to engage in the research process, diagnostic tools, or professional development tools that help educators implement research-based practices. RELs may also design new interventions (programs or practices) in areas where there are few or no effective existing interventions available; these interventions should be informed by extant evidence and applicable theory.

· Each REL contractor is required to develop a toolkit based on a What Works Clearinghouse practice guide to help educators use evidence-based practices in classrooms. Each toolkit will include: (1) Initial Diagnostic and On-going Monitoring Instruments (to assess both teachers’ practice of and school-/district-/institution-level implementation of the practice guide’s recommendations), (2) Professional Development Resources that help an educator or a community of educators acquire the knowledge and skills associated with the recommendations and how to implement them, and (3) Steps for Institutionalizing Supports for Evidence-Based Practice. RELs must also evaluate the efficacy and implementation of the finished toolkits to examine changes in teacher practice and learner outcomes. The practice guides that the toolkits will be based on are:

· Teaching Math to Young Children (REL Appalachia)

· Teaching Strategies for Improving Algebra Knowledge in Middle and High School Students (REL Central)

· Teaching Elementary School Students to Be Effective Writers (REL Mid-Atlantic)

· Developing Effective Fractions Instruction for Kindergarten through 8th Grade (REL Midwest)

· Assisting Students Struggling with Mathematics: Intervention in the Elementary Grades (REL Northeast & Islands)

· Using Technology to Support Postsecondary Student Learning (REL Northwest)

· Teaching Secondary Students to Write Effectively (REL Pacific)

· Assisting Students Struggling with Reading: Response to Intervention (RtI) and Multi-Tier Intervention in the Primary Grades (REL Southeast)

· Providing Reading Interventions for Students in Grades 4–9 (REL Southwest)

· Improving Reading Comprehension in Kindergarten Through 3rd Grade (REL West) The REL Program uses a SharePoint site to facilitate the review process. Relevant contractor staff will be given access to the site so that they can receive email notifications when a product is ready for review or editing, submit review teams for NCEE approval, download documents for review or editing, and submit completed reviews, edited products, and final products.

VII. Task Requirements

This section specifies the tasks and subtasks that the contractor shall perform.

Task 1: Communication with IES

1.1 Security Clearances

All positions will be designated as either no risk, low risk, or moderate risk level. Within the first week of the contract start date the contractor shall appoint a security liaison who shall submit a list of all staff on the contract with the appropriate position risk level designation. The security liaison shall submit the required documentation for all staff assigned a low or moderate risk to the IES security team within two weeks of contract start date, and as needed for staff added over the life of the contract. The security liaison shall be responsible for submitting the required documentation for staff from the prime contractor and any subcontractors. Additionally, staff shall complete periodic cybersecurity and records management training provided by IES.

1.2 Kick-off Meeting

Within 1 week of award, the Contractor shall participate in an opening kick-off meeting with the COR, contract specialist (CS), and NCEE staff to discuss plans for the activities to be performed soon after the award date. Within 1 week of this meeting, the contractor shall prepare a short (3-5 page) summary memo on the key issues addressed and any follow-up recommendations or planned actions.

1.3 Communication with IES

The contractor shall participate in regular telephone calls with the COR to discuss issues related to the review contract, including any anticipated problems before they occur and proposed solutions to those problems. These calls shall occur every 2 weeks. The contractor shall send an agenda for the call at least two working days before each call. Within five working days following each meeting, the contractor shall prepare a memo summarizing the key issues and concerns raised at the meeting and how each will be addressed.

The Director or Deputy Director shall communicate with the COR on a regular basis, such as by phone or email, to discuss urgent points and follow-up as needed.

The Director and Deputy Director shall meet with the COR quarterly to discuss feedback on the quality of completed work and plans for future work.

1.4 Monthly Progress Reports

The contractor shall submit monthly reports containing the following: 1) information on the progress on each task and activity, including accomplishments and problems that have been resolved or are in need of resolution; 2) information concerning expenditures and other relevant budget related details; and 3) anticipated activities and deliverables for the upcoming month. These reports shall be submitted by the 10th business day of each month.

In the first monthly reports submitted after the end of each contract year (corresponding to months 13, 25, 37, 49, and 60), the reports should also quantify the work accomplished over the previous year, including: the number of documents reviewed by product type; average number of times each document was reviewed by product type; average number of days to complete reviews; number of as-needed guidance calls that were conducted with RELs; number of reports that completed editing and the average number of days to complete editing; number of reports that completed production and the average number of days to complete production.

Task 2: Refine and Implement Review Criteria and Processes for REL Work

2.1 Refine the Existing Criteria for Review

The contractor shall refine the existing review criteria and procedures for each type of REL product as needed based on NCEE’s needs, the RELs’ needs, and to keep up with new methods and practices in the field. Review criteria shall be written in a fashion that will make them user-friendly for REL staff and shall address relevance, technical quality, readability, and utility (see below for more information on these review categories). The review criteria shall take into account all existing IES and NCEE standards. Revisions may be requested by NCEE, and the contractor shall also propose revisions when needs or potential improvements are identified.

Some revisions to the review procedures may require changes to the SharePoint system used to facilitate the review process. IES will work with the REL Website Management contractor who administers the SharePoint site if changes to the site workflows or other features are necessary.

2.2 Review Research Proposals and Products

The contractor will review project proposals for research projects (to include descriptive and correlational data analyses, causal impact studies, efficacy studies, and systematic literature reviews and meta-analyses) as well as tools, toolkits, and new REL-developed interventions. The contractor shall also review the products that result from these proposals. Over the five-year period of this contract, NCEE anticipates up to 200 REL research proposals, plus up to 200 REL reports/products that result from those proposals, will require peer review. The number of proposals and products that will begin review each year of the contract will vary over time. NCEE estimates that the volume of proposals and products submitted for review will vary by contract year by approximately the percentages presented below. As the REL contracts began in 2022, there may be 30-40 proposals that started the review process in 2022 before the REL Peer Review contract starts in 2023.

For each REL research proposal and product submitted for peer review, the contractor shall identify a team of senior-level policymakers, practitioners, and researchers who can evaluate its relevance, technical quality, readability, and usability. This will include adherence to all IES standards. The categories for review are described below. Review teams must have expertise in three areas relevant to the REL product being reviewed: methodology, content, and practice. The name and qualifications (in the form of a short biographical summary) of each reviewer for any given product must be presented to the COR within two business days of receiving the proposal or product to be assessed. Should new reviewers be requested by COR, a revised set of reviewers must be provided by the contractor within three business days.

The contractor shall submit to COR written review comments for each proposal or product within two weeks of receipt of the proposal or product. In most cases, the contractor will be asked to review proposals or products two to three times as they are revised and improved by REL staff. The contractor must have a system for tracking the issues from previous reviews that were successfully addressed and those that require additional attention. The review shall address the criteria described below. Comments related to a proposal or product shall be presented to COR through careful use of comments in Microsoft Word, as well a detailed memo providing overall comments related to broader issues of strengths or weaknesses in a document. All comments and memos shall be written constructively with the aim of helping authors correct or address any deficiencies in the REL product while also helping to build the knowledge and capacity of the author to prevent similar errors in the future. Review memos shall include recommendations about whether or not a proposal or product should move forward to implementation or publication.

Review Criteria

Relevance The contractor shall provide written reviews on the relevance of project proposals and products. For project proposals, the review memo shall assess the extent to which the proposed end product has the potential to provide actionable information to a particular audience, the extent to which similar information is or is not already accessible, and the degree to which any potential findings will be worth the data gathering burden imposed on any public entities. The review memo shall also consider the extent to which a rationale has been presented for the work including through the provision of evidence from a needs assessment or input from the partnerships of which the REL is a part.

For reports or products, the section of the review memo dealing with relevance shall assess the extent to which the evidence provided is actionable, the degree to which the presentation of research findings is appropriate for a practitioner/policymaker audience, and whether time consumed reading the research product is balanced by potential benefit or learning.

Individuals charged with reviewing the relevance of REL proposals and products must be individuals who would be likely to use REL products; such individuals can include current and former school district leaders, policymakers and program directors from the state and district levels, classroom teachers with more than ten years of experience, leaders in the education industry, and leaders of not-for-profit organizations focused on education advocacy, program provision, and support.

Technical Quality The contractor shall provide written reviews on the technical quality of research proposals and products. The written reviews shall provide feedback on the extent to which REL proposals and products meet the standards and expectations of IES and NCEE related to research design, data collection, data analysis, and reporting of findings. For project proposals, the review memo shall assess the strengths and weaknesses of the proposal and the feasibility of completing the proposed work in the timeline proposed. This will include review of the technical approach used to meet NCEE standards, sampling and statistical inference methods, data sources or outcome measures to be used, and analysis plans.

Depending on the scope and content of the research proposal, review will also include assessment of the proposed methodology, recruitment of study subjects, issues related to sample size and minimum detectable effects, instruments to be used, and strategies for dealing with attrition and missing data. The review will also provide recommendations for strengthening any of these aspects related to the design or analysis proposed in project proposals.

Individuals charged with reviewing the technical quality of REL proposals and products must be senior level researchers or analysts who have demonstrated the ability to design, manage, and lead high quality descriptive and evaluative studies in the field of education. Individuals who will be reviewing evaluation studies or methods papers must have extensive familiarity with evaluation methods and IES standards for scientifically valid research and evaluation. This shall include familiarity with the evidence and reporting requirements of the What Works Clearinghouse. In addition, reviews of technical quality may require psychometric expertise in some cases.

Readability/Writing Quality and Style The contractor shall provide written reviews of the readability and writing quality for all REL products and reports. (Reviews of readability are not requested for proposals.) The reviews shall provide feedback on the extent to which REL products are written in a clear and engaging fashion. Reviewers shall comment on the writing style of all products and the extent to which they can be understood by a practitioner or policymaker. (See the Federal Plain Language Guidelines.) Review comments focused on readability shall help the authors to eliminate redundancies; perfect word and syntax choices; structure paragraphs and reports in a logical manner; and create products that are easily consumed by their intended audiences.

Individuals charged with reviewing the readability of REL products must have demonstrated expertise in the clear and careful written presentation of research findings for diverse audiences. In some cases, reviews related to readability may be conducted by the same individuals reviewing products for relevance or technical quality while in other cases, reviewers for readability may be other individuals who are expert in issues of writing and presentation.

Utility The contractor shall provide written reviews on the utility of tools, toolkits, interventions, or other products intended for use by researchers, policymakers, or practitioners. The written reviews shall provide feedback on the extent to which REL products are able to be used by their intended audiences. Reviews of utility shall be conducted on proposals and products. For proposals, reviews related to utility will provide written comments on the extent to which any tools, toolkits, interventions, or other products are likely to be in a form, and inclusive of content, that could be easily used by the intended audience. Reviews for the utility of products shall include an assessment of the likelihood that the intended audience will benefit from and be able to use the product as designed.

Individuals charged with reviewing the usability of REL products must represent experiences similar to those of the intended audience for a REL product. For example, products that are targeted toward district administrators shall be reviewed for usability by individuals who have experience working in school district central offices. Products that are targeted at researchers shall be reviewed by researchers with experience in a similar field. The range of experiences that will be needed for usability reviews will include school and district leadership, state education agency leadership, and researchers with significant quantitative and qualitative research experience.

2.3 As-Needed Guidance for Improvement of REL Products

During the peer review process, NCEE and REL staff will occasionally require additional guidance and consultation about how to improve particular proposals and products. The needed guidance can be related to any of the four review criteria described above (i.e., relevance, technical quality, readability, and utility) and shall be provided in telephone/virtual conversations between the contractor’s reviewers and REL and NCEE staff members. The guidance will most often focus on methodological issues (technical quality) related to research reports. Less frequently, guidance may be needed about other issues related to REL products.

Telephone calls/virtual conversations for as-needed guidance will take place approximately 5-10 times per year and will each last approximately one hour. The contractor will provide a call-in number for participants to use and will be responsible for scheduling the call at a time that is acceptable to all participants (REL staff, NCEE staff, and reviewers). Within a week of the call, the contractor will provide a written memo summarizing the content of the call, the guidance provided, and recommended next steps for the REL authors to take in order to revise the proposal or product.

Task 3: Editing and Production of Reports

3.1 Update the “REL Writers and Style Guide” and Report Templates

The contractor shall, as needed, update the “REL Writers and Style Guide” which provides guidance to the REL authors about how to prepare a report that will meet IES and NCEE requirements for report structure and writing quality. The guide also provides style guidelines for references, headers, footnotes, table formats, etc.

As requested by NCEE, the contractor shall also update or develop new REL report templates that are designed to be used for various REL report types. The templates shall adhere to IES style guidelines and demonstrate appropriate use of the REL Writers and Style Guide guidance.

3.2 Report Editing and Production

IES requires a way to ensure report quality and consistency across the 10 RELs. REL products must follow the same organizational structure, and have consistency in layout, handling of footnotes, references, tables, statistical nomenclature, and other style guidelines.

Over the five-year period of this contract, NCEE anticipates up 150 REL reports/products will require editing and production. This is fewer than the number of products that will be peer reviewed because RELs have the option to release some research products in alternate, non-report formats (such as PowerPoint presentations), which will not need editing or production. The number of products that will begin editing each year of the contract will vary over time. NCEE estimates that the volume of products submitted for editing and production will vary by contract year by approximately the percentages presented below.

The contractor will be provided with a Word file for each report or tool, in the standard REL report template, as well as keywords, alternative text describing figures and equations, underlying data for figures, vector-editable files for maps, an image for the cover photo, and text for an IES Newsflash announcement that will be released when the product is published.

The contractor shall conduct a technical edit assuring that all grammar and other conventions for written research are met, as well as revising the product as needed in order to adhere to the Writers and Style Guide requirements. The contractor shall have two weeks to fully edit the Word document. The edited document, showing the tracked changes and including any questions from the editors, will be sent to the REL for review. The Newsflash shall also be edited and sent to the REL. Once the REL’s feedback has been incorporated, the contractor will submit a clean version of the product for final review by NCEE. NCEE will review the edited document and provide feedback if needed.

The contractor shall then prepare the product files in a design template. RELs typically submit one combined Word file for review and editing that contains the main report file, the appendices, and one or more summary documents. The contractor will be responsible for separating these files and formatting them into a report file, an appendix file, and one or more summary files. Note that REL tools and toolkits (as opposed to research reports) vary widely in content and structure, and may require the production of just one file or many files. The REL and NCEE will then each review the draft PDF files. The contractor shall then make any necessary modifications and deliver the final PDF files, which must comply with Section 508 of the Rehabilitation Act (click this link for further information).

As part of the editorial review, the contractor shall:

· Ensure tone is appropriate for the target audience;

· Ensure a logical presentation of report findings, or of tool instructions and components;

· Ensure the proper use and placement of tables, charts, graphics, headers, bullets, “call outs,” etc.,

· Be responsible for overall copyediting and proofreading to ensure there are no errors and that a style consistent with NCEE guidelines is used for citations, tables, footnotes, references, and title and verso pages.

As part of the production process, the contractor shall:

· Format all product files from Word to the design templates;

· Ensure design and layout meet 508 standards;

· Provide final PDF files.

Task 4: Webinars for REL Staff

On rare occasions when needed, the contractor will host and present webinars for REL staff to announce and explain any major changes to the review criteria, the Writers and Style Guide, and/or the review or production processes. One or more webinars will also be presented in early 2027, once the 2027 cohort of REL contracts begin, to introduce and/or re-familiarize the new REL contractors with the review criteria, review process, the Writers and Style Guide, and the report production process. Webinars will each last approximately one hour and may include up to 100 participants. The contractor shall provide and facilitate the webinar platform and record each presentation.

Task 5: As-Needed Analytic and Methods Oriented Projects (Optional Task)

As needs arise, NCEE may require the contractor to design, conduct, and report the findings of research or evaluation activities designed to serve as exemplars for the REL program. These activities are distinct from the guidance described above in that they produce a “worked example” of how a REL might address a particular analytic challenge.

Examples of projects could include:

· Contrasting differing approaches to addressing missing data and creating a “primer” on those that prove most promising;

· Identifying opportunities in planned studies conducted by the RELs or by other NCEE contractors to test innovative methodological approaches; and

· Conducting replication studies that compare the results of non-experimental methods with experimental methods.

5.1 Analytic and Methods Oriented Projects

Once NCEE initiates a new project, the contractor shall produce a project workplan that details the main goals of the project, the questions that will be answered, and the expected benefit of the project to the REL program, education researchers at large, and NCEE.

After NCEE has approved the workplan, the contractor shall carry out the work. The contractor will be responsible for any OMB clearance that may be required related to the approved project. Interim updates shall be shared with the COR in monthly progress update memos throughout the project life cycle.

The resulting products shall comport with prevailing IES technical and stylistic guidance and shall be appropriate for the intended audiences. All products will be initially reviewed by the COR and NCEE leadership.

Any field-facing product must also be reviewed by a panel of relevant experts and stakeholders via a Technical Working Group (see Task 5.2). After incorporating revisions in response to the recommendations made by the panelist, the contractor shall submit a revised version of the product to their COR.

Once a field-facing product has received final approval by the COR, it will be submitted for peer review to IES’s Standards and Review Office (SRO). Review by SRO can take up to four to six months. The contractor shall submit revised drafts to SRO within 4 weeks of receipt of SRO comments. The contractor should expect at least 2 rounds of SRO review. Once the report is approved by SRO, the contractor will format and submit the final report for release.

5.2 Technical Working Group Meetings

The contractor shall organize and convene a meeting of experts to review products developed under Task 5.1 before final submission to NCEE.

The contractor shall work closely with the COR to identify and propose a suitable list of between 4 and 6 experts to participate in the meeting. These experts will review draft reports/products, assure they meet scientific standards, and recommend changes where necessary. The contractor shall assess whether potential invitees may have any conflict of interest pertaining to the outcomes of the study and provide the list of proposed invitees to ED for approval. Upon COR approval of the experts, the contractor shall invite these individuals to participate in the meeting.

The meeting(s) shall take place virtually or in Washington, DC at facilities provided by IES. The contractor shall provide remuneration to the experts participating in such meeting(s) at a rate not to exceed the current hourly wage of a federal employee who has achieved Grade 15, Step 10 on OPM’s General Schedule (adjusted for the Washington, DC locality; see https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/ for current rates).

Remuneration shall take into consideration the amount of time spent participating in the meeting, preparing for it, and for completing any post-meeting obligations. The contractor will be responsible for all travel costs associated with these meetings (both their own and all participants) and shall coordinate all logistics related to these meetings. Federal funds may not be used to provide food.

The contractor shall prepare an agenda and any needed materials for each meeting held. Within 1 week of each meeting, the contractor shall prepare a short (3-5 page) memo that summarizes the feedback provided by the expert panel and details the planned revisions to the report/product.

Task 6: Pilot Data Collection Methods for Examining the Use of Research Evidence (URE) (Optional Task)

This optional task will help the RELs pilot new Stakeholder Feedback Survey (SFS) items and/or to pilot other methods for collecting data on how the RELs’ partners and other participants understand and use the research and evidence-based practices that they learn about from the RELs.

The purpose of the piloting of additional items and other data collection strategies is to answer the following questions:

1. Are there survey items and data collection strategies available in the William T. Grant Foundation’s “Use of Research Evidence (URE) Repository Methods” at https://uremethods.org/ that could be added to the SFS to enable the RELs to study how their partners and participants use the research and evidence-based practices the RELs provide?

2. Which survey items and data collection instruments would be most appropriate for the RELs to use if they want to undertake research that answers questions such as:

a. How do REL partners and participants in REL activities report using research evidence and evidence-based practices before partnering with the REL, and do they report changing in these areas during or after involvement with REL work?

b. If REL partners and participants change the way that they use research evidence and practices, in what ways do they change? Do they use evidence and evidence-based practices more frequently? Do they report having a better understanding of how to judge the validity of research evidence? Do they use evidence to inform decisions? Is there evidence that their understanding of research on a issue has informed their conceptual thinking about the issue? Do they share their understanding of research and evidence with colleagues informally or in formal ways?

c. What characteristics of REL support do partners and participants report being most helpful and effective in helping them to understand and use research evidence?

d. Do REL partners and participants engage in other, non-REL activities and practices that help them to increase their skills in understanding and using research? Does their contact with the RELs appear to enhance their ability to participate or benefit from those other non-REL activities and practices?

e. Does the information that the RELs collect through the additional SFS items or other data collection strategies provide data that helps the RELs improve the quality and effectiveness of their work with education stakeholders?

6.1: Identify URE Experts

Within two weeks of the award of this optional task, the contractor shall propose a list of experts on “Use of Research Evidence” (URE). The contractor shall propose a final list of experts within two weeks of receiving feedback from IES.

6.2: Assess and Pilot Data Collection Methods

The contractor shall work with a subgroup of RELs to assess the validity and reliability of required SFS items and pilot new SFS items and other possible data collection strategies for learning about the use of research evidence among REL partners. Within eight weeks of the award of this optional task, the URE experts shall submit a draft plan for working with a subgroup of RELs[footnoteRef:2] to do the following:

[2: After award of the contract, IES will provide the contractor with a list of RELs that are interested in helping with this work.]

· Assess the validity and reliability of the SFS items that IES requires the RELs to administer to stakeholders after participation in REL Training, Coaching and Technical Support activities and to REL Partnership members at the end of each program year;

· Pilot new SFS survey items and other possible data collection strategies for learning about the use of research evidence among REL partners.

The plan shall outline a strategy for collecting data from a sufficient number of REL partners and participants in order to assess reliability and validity and answer the questions listed above using new survey items and other data collection methods. Within two weeks of receiving comments from IES on the draft plan, the contractor shall submit a final plan.

The contractor shall also develop an OMB clearance package for the collection of data from respondents. Within four weeks of the submission of the final plan to IES, the contractor shall submit the draft OMB package. Within two weeks of receiving IES comments, the contractor shall submit a revised OMB package. Within two weeks of receipt of public comments on the package after the 60-day Federal Register notice, the contractor shall submit a revised package that addresses the comments. Within two weeks of receipt of public comments and comments from OMB after the 30-day Federal Register notice, the contractor shall submit a revised package that addresses all of the public’s comments and OMB’s comments.

The work outlined in the plan shall take place over no more than a 24-month period and shall result in a memo to IES that summarizes the findings and outlines recommendations for SFS items that are valid, reliable, and can be used by the RELs for conducting research on the use of research evidence. The URE experts and the contractor shall also present their findings and recommendations through a virtual meeting with IES.

Deliverables:

Task 1: Communication with IES

1.1 Security Clearances

1.2 Kick-off Meeting

Summary of kick-off meeting

1.3 Communication with IES

1.4 Monthly Progress Reports

Task 2: Refine and Implement Review Criteria and Processes for REL Work

2.1 Refine the Existing Criteria for Review

2.2 Review Research Proposals and Products

2.3 As-Needed Guidance for Improvement of REL Products

Task 3: Editing and Production of Reports

3.1 Update the “REL Writers and Style Guide” and Report Templates

3.2 Report Editing and Production

Task 4: Webinars for REL Staff

Task 5: As-Needed Analytic and Methods Oriented Projects (Optional Task)

5.1 Analytic and Methods Oriented Projects

5.2 Technical Working Group Meetings

Task 6: Pilot Data Collection Methods for Examining the Use of Research Evidence (URE) (Optional Task)

6.1 Identify URE Experts

6.2 Assess and Pilot Data Collection Methods