REL_-_RFI_Addendum_3_-_01.06.16.docx

REGIONAL EDUCATIONAL LABORATORY

FY16/FY17 ACQUISITION

Solicitation No.: ED-IES-15-R-0016

RFI Addendum 3: Response Feedback & Additional Information

Facilitated by the Office of the Chief Financial Officer, Contracts and Acquisitions Management Office

Table of Contents

Introduction

The purpose of this document is to address competition questions submitted in response to the Department’s Request for Information (RFI) issued on August 7, 2015, as well as questions that potential offerors have submitted in the interim. This addendum builds on earlier addenda posted to FBO and provides updated draft competition information including but not limited to the following documents: (1) Performance Work Statement (PWS); (2) Instructions to Offerors; and (3) Evaluation Scale. Please pay close attention to the following changes:

(1) The PWS included in this document is a marked-up version that shows changes made after the August 7th RFI posting.

(2) The instructions to offerors generated strong discussion, and we have revised the instructions to both reflect the needs of the REL program and to respond to concerns from the vendor community about perceived burdensome requirements. This document does not contain track changes.

(3) We have clarified our procedures regarding subcontracting arrangements for this competition. We have included a draft small business evaluation tool which will provide a quantitative method to evaluate an offeror’s proposed substantive small business participation. We intend to use this tool during the competition, but note that the weights provided in the tool are for example purposes only and do not necessarily reflect the weights that the Department will use in the competition.

(4) We have included a draft price matrix as a separate Excel attachment, and we have provided guidance on our expectations for the matrix at page 86.

(5) We have included additional guidance for the REL websites that each successful offeror will create and maintain during the next contract cycle.

(6) Please note the following documents that are posted with this addendum:

a. Excel Price Matrix

b. Wireframes for individual REL websites

c. Small Business Calculation Tool

We also take this opportunity to acknowledge that Congress has passed, and the President has signed, the Every Student Succeeds Act (ESSA) – a reauthorization of the Elementary and Secondary Education Act and the successor to No Child Left Behind. ESSA may affect how states and districts prioritize the types of support they seek from the RELs. However, the Department believes that the tasks described in this PWS encompass the types of assistance that states are likely to request as a result of ESSA.

To the extent that the contents of this document raise additional questions or require further clarification, please submit questions directly to Jonathan Bettis (Jonathan.Bettis@ed.gov) no later than January 11, 2016 by 3:00 PM EST. This will be the final opportunity to ask questions prior to release of the solicitation.

Offerors may submit questions via email or a separate Word document, or offerors may include comments or track change revisions to this addendum if doing so makes the comments or revisions easier to convey with proper context.

Please be advised that the terms and conditions of the final solicitation control.

We will not respond to additional questions before we release the solicitation as we remain committed to a January 30, 2016 release.

Vendor Questions and Responses

Contract Structure

Many respondents requested that we designate certain tasks as Cost Reimbursement (CR) rather than Firm-Fixed Price (FFP). We agree that certain tasks that we originally designated as FFP are better suited as CR, and we revised the task designation accordingly. We decline to designate the contracts as entirely FFP even though the current cohort has successfully performed under this contract structure.

PWS: Framing

Comment: With regard to the following language “within a short time horizon.” The approval process would need to be streamlined in order to meet the short-term timeframe. We recommend stating “once the steps within the approval process have been consolidated, RELs should complete tasks within a short term horizon.”

Response: We disagree. The length of the approval process for projects is directly related to the quality of the plan that has been submitted. In addition, RELs have many tools at their disposal beyond conducting research studies that they can use to meet stakeholder needs in a short-term timeframe, including training, coaching, and technical support meetings, events, and tools.

Comment: Response to "To the extent possible, REL activities shall be recorded, archived, and posted in a user-friendly manner (for example, by breaking a long presentation into shorter segments) for later viewing by those who could not attend the event in-person. Videos from webinars and events held by the 2012-2017 RELs are posted on IES’s YouTube site." From our experience, it’s best to have fewer high quality webinars versus a higher quantity of webinars. Stakeholders have provided feedback that they are overwhelmed by a large quantity of webinars.

Response: We agree that RELs should provide high-quality webinars. The PWS language reflects our belief that stakeholders will not watch a long (e.g. 2.5 hour) webinar.

Comment: The RFP should consider how IES’ structure might support flexibility and creativity, which the current approval process is not always conducive to promoting. We need quality controls, but we don’t want the review process to stifle creativity, flexibility, adaptation, and partner engagement throughout the process.

Response: The review process ensures that products meet the high standards of IES. Creativity, flexibility, adaptation, and partner engagement are possible within the current structure.

Comment: Do single case designs and regression discontinuity designs also fall under this category (i.e. meets What Works Clearinghouse Standards without reservations)?

Response: According to the WWC Handbook, randomized controlled trials (RCTs) and regression discontinuity design (RDD) studies are the only individual studies that can meet WWC standards without reservations. Individual single case design (SCD) studies do not receive an evidence rating according to WWC standards. Instead, individual SCD studies receive a rating according to WWC design standards. A combination of 5 SCD studies, conducted by 3 sets of researchers, including a combined sample of at least 20 students is required for a set of SCD studies to be reviewed against WWC evidence standards. In contrast, individual group design studies are reviewed against both WWC design and evidence standards.

Comment: Can IES provide examples of how previous and/or current RELs have built capacity for the education research sector

Response: No. We prefer each offeror develop its own examples.

Comment: Based on the past five years of experience, is there any indication of the “ideal” size of research alliances?

Response: There is no “ideal” size for an alliance. Size depends on a number of factors and will vary region to region based on the purpose of the alliance.

Comment: In response to "the REL work will with at least 2 research alliances": this seems somewhat low.

Response: Offerors may propose more than two alliances.

Comment: Are the alliances currently fixed over the entire life of the REL? It might make sense for some partnerships to be shorter-term and evolve in focus (to be more responsive to changing needs of SEAs in particular). Is this allowable? Does this imply then, as the above comment addresses, the potential for partnerships to NOT be fixed over the life of the REL?

Response: This is permitted. Partnerships do not necessarily need to be fixed over the life of the REL.

Comment: Is it expected that the partnerships are established from the beginning of the current REL cycle, or is it allowable to develop some partnerships during the REL cycle?

Response: Partnerships can be established throughout the life of the REL contract. The formation, composition, duration, and conclusion of each partnership depends upon the nature of the work.

Comment: The PWS includes the following statement: “For the program to achieve this goal, there must be widespread recognition among regional education leaders that the REL is a trusted and capable resource for identifying, producing, and supporting their use of research.” Does the Department have any plans to assess or measure this?

Response: The Department will evaluate the entire REL program. This evaluation is performed under a separate contract and will likely begin in the second year of the REL contract.

Comment: Will there be coordination by IES or a coordinating entity if several RELs want to develop collaborations with the same non-partisan national entity?

Response: It is reasonable that a non-partisan national entity may partner with several RELs. The 2017-22 REL contracts will not have a third-party coordinating entity. Coordination is a key element of the REL PWS, and we expect each REL contractor to develop collaborations/coordinate as appropriate.

Comment: What are the criteria for literature reviews that would be approved under the new contracts?

Response: The current guidelines for literature reviews can be found beginning on page 25 of the document “NCEE Guidance for REL Study Proposals, Reports, and Other Products (April 2013)” that is available for download at this link: http://ies.ed.gov/ncee/edlabs/relresources. Literature reviews in the next REL contracts will likely be required to follow similar guidelines.

Comment: It seems that RELs should provide research evidence to an SEA considering a new policy to help them use research to shape the policy. Would this be within the REL mission?

Response: Yes, this is within the scope of the REL program.

Comment: What is significant demonstrated scholarly expertise? What is demonstrated expertise? Reports typically have multiple authors, is this expected of all authors? Can a different structure for engaging experts besides authorship be considered, such as an advisory committee? (“Authors of REL reports shall have demonstrated expertise in the content area being addressed by the report, ideally shown through a substantial portfolio of published work in that area.”)

Response: This is clarified in the PWS to indicate that lead authors should have demonstrated scholarly expertise.

Comment: Can an offeror propose to create a pool of resources to draw upon unnamed experts who will be called upon to address new regional needs as they arise?

Response: Offerors are expected to propose content experts, as both named individuals and firms or universities with demonstrated expertise. It would be impossible for the Department to evaluate the level of content expertise an offeror has to pull from without having specific individuals and/or firms named.

Comment: Agenda implies more than one piece of work with a partner, however, in the next bullet it is implied that a partnership can have one or more projects. Is an agenda with one substantial project acceptable?

Response: The REL shall support a coherent agenda (or agendas) of work. This may take place across multiple partnerships and partnerships may have one or more projects.

Comment: Regarding the following two elements:

3. Widespread recognition among the region’s state and local education agency leaders of the REL as a key resource for credible research and support for using research in education. The REL program is the Department’s primary resource for supporting state and local education agencies to use research evidence in education decision making. For the program to achieve this goal, there must be widespread recognition among regional education leaders that the REL is a trusted and capable resource for identifying, producing, and supporting their use of research.

4. Increased individual and organizational capacity to access, understand, interpret, critique, apply, and/or conduct research, particularly at state education agencies in the region. A key required outcome for the REL is a demonstrable increase in the individual and organizational capacity of education entities in the region to use research evidence in education decision making. State education agencies in the regions are [one of the] key clients for this federally funded program, and therefore the Department expects, at a minimum, that there will be improvements in capacity at the state level

How will these be measured?

Response: As indicated in the PWS, these long term outcomes are provided as a sample for how the Department is thinking about the outcomes of a sustained long-term investment in the REL Program and the outcomes that will result if the investment strategy is successful. The first REL awards were made in 1965, so after 50 years of federal funding, these goals should be within reach. Every new REL contract-holder is standing on the shoulders of the previous REL organizations, staff, projects, and relationships. All offerors should organize their REL proposals with the cumulative and collaborative nature of the program in mind.

PWS: Topical Priorities

Comment: It is unclear whether school and district leadership is a priority topic. We suggest including it as one.

Response: We do not believe that this is necessary. The PWS includes many possible priority topics for offerors to choose from as it relates to the needs of their region. Offerors may also propose their own priority topic.

Comment: Does IES consider educator effectiveness/educator quality a high leverage education problem?

Response: The answer depends on the regional context. As described in the PWS, high-leverage problems are those that (1) if addressed could result in substantial improvements in education outcomes for many students or for key subgroups of students; (2) are priorities for regional policymakers (particularly at the state level); and (3) require research or research-related support to address well.

Funding:

Comment: Is it correct to assume that any budgeted funds for FFP tasks/subtasks will not/cannot be carried forward to the next contract period? Similarly, can budgeted funds for CR tasks/subtasks be carried forward each contract year? Are the CR funds cumulative and can they be used only for that task/subtasks or may those funds be “redistributed” in the new budgets prepared each year?

Response: The Education Sciences Reform Act provides specific direction about the duration and structure of the REL program. For this reason, funding allocated for the RELs is somewhat different from other IES programs. The REL contracts will be multiyear contracts, and funding obligated in one year may cross Fiscal Years for planning and invoicing purposes.

The final contract will likely separate Cost Reimbursement (CR) funding from Firm-Fixed Price funding (FFP). Furthermore, FFP funding will be further subdivided between Information Technology (IT) and Non-IT Contract Line Items (CLINs). CR funds may not be used to pay for FFP deliverables and vice versa.

Comment: What level of cost accounting detail will the Department want each month on vouchers submitted for CR tasks/subtasks? What level of cost documentation will need to accompany these invoices? For example, we assume Labor will need to be presented by individual and include hours worked and hourly rate. Are percentages of time worked, without documentation of actual timesheet hours acceptable? Will plane tickets and receipts for expenses over $15 have to be submitted or copies of monthly phone bills, Xerox charge, software licenses charges, etc., have to be attached?

Response: The Department expects awardees to include labor hours expended as well as percentages of time worked for each task. We do not expect you to provide time sheets or other documentary proof of actual hours worked. Awardees should include receipts for travel costs when invoicing travel costs. We do not expect companies to include monthly phone bills, Xerox charges, licensing fees, etc. in their monthly invoices. Of course, companies should include these costs in their business proposals during the solicitation phase. Many of these costs will factor into a company’s General and Administrative (G&A) rates while others may appropriately be classified as other direct costs. The Department has the right to request additional documentation when reviewing invoices if necessary.

Comment: Currently, the Instructions ask the Offeror to confirm that the quoted costs in the budget are good for 300 days. This type of confirmation is typically requested for 30 to 90 days. Given the long period of confirmation how do we address changes that may occur in labor costs or in our Provisional Indirect Cost Rates?

Response: We are unable to alter this date for a variety of reasons. First, given the scope of the REL program our technical review process will span the remainder of FY16 and will likely extend into FY17. For this reason, we require offers to remain valid for enough time to cover the entire technical review period. We do not believe that this is unreasonable. As an offeror, you must anticipate labor costs over the life of the contract. We would not expect to see substantial variation in rates over a six-month review period. If a company’s Provisional Indirect Cost Rates change from the time of submission to award, we expect companies to inform us of the change. We would encourage offerors to work closely with their auditing agency (DCAA, HHS, etc.) to ensure that this information is timely and current.

Comment: Can the Department clarify what type of incentives, if any, will apply?

Response: We anticipate awarding a Cost Plus Fixed Fee contract. We are still considering what incentives – if any – may apply. It is unlikely that we will include any award fee in the contract.

Topic Papers

Comment: The other issue is that a number of the incumbent RELs have been working on several of these topics in ways called for in the three Concept Papers. Even with random assignment there may be the perception that some REL regions received topics that were favorable to them. Would the Department consider modifying this process by still using the randomized assignment of a topic but also requiring an Offeror to select one of the remaining topics and prepare Concept Papers for that topic? We believe Offerors would view this as a fair process. While it could be argued that such a change would put pressure on the 75 page limit, an experienced Offeror knows how to adjust page limits among requirements based on importance. The Department might also consider increasing the page limit to 100 pages.

Response: Offerors have many opportunities to demonstrate their ideas, expertise, and plans related to their preferred topics within the content of the proposal. Because successful REL contractors must be able to respond to regional needs across many content areas, we believe it is fair to randomly assign the topics for concept papers that will be included in Appendix A. All offerors for the same region will provide concept papers for the same topic. The page limit will be 75 pages.

Comment: Does interrater reliability become more of an issue using an adjectival rating method as opposed to point value? If so, how does the Department plan to address the issue?

Response: The Department has experience using adjectival ratings when evaluating proposals and this approach is appropriate for the current competition. There is no difference in the quality of the proposal review process, panel discussion, or award decision when using this approach.

Comment: Can the Department provide additional explanation of the TEP review process?

Response: As a general rule the Department does not disclose details regarding specific source selection procedures other than information that is properly subject to release under FAR Part 15. A panel of reviewers will review each compliant proposal and will evaluate that proposal against the stated criteria. The panel will convene to discuss each proposal individually against the stated criteria, and the acquisition will proceed in accordance with the procedures outlined in FAR Part 15 (e.g. establishment of a competitive range if applicable, discussions/negotiations, final award decision, etc.).

Key Personnel

Comment: Will the Department consider allowing more resumes or increasing the page length of the CV?

Response: Not at this time. We feel that the current resume limitations provide sufficient information to allow the Technical Evaluation Panel to determine whether an offeror proposes the proper personnel with the requisite experience for the requirement.

Comment: This is a high percentage of time for a senior person (Deputy Director) and could have the consequence of preventing strong candidates from serving in the deputy director role. Consider reducing the percentage of time required.

Response: The REL program requires the highest level of experience for the Director and Deputy Director position. We also require those individuals with experience to devote the majority of their time to the REL program. We cannot risk a company proposing a qualified individual only to have that individual devote substantially less time to the REL than proposed. That being said, we have reduced the required minimum percentage of time. The Director must spend a minimum of 60% time, and the Director and Deputy Director together must devote the equivalent of at least 1.4 full time equivalent positions to the REL contract. The distribution of percentage of time across the Director and Deputy Director can be proposed by the offeror. The two positions may not be held by more than two people to add up to the 1.4 FTE.

Comment: Can IES provide criteria for the Task 3 lead?

Response: Offerors shall use their expertise and judgement to provide a qualified Task 3 lead.

Comment: Given the breadth of this task, would IES allow a contractor to staff co-leads for Task 5 (i.e., 30% FTE for two key staff) – assuming there are clear lines of authority specified by REL leadership?

Response: The Department strongly recommends having one task lead because past experience has demonstrated that co-task leads tend to create work that is siloed instead of integrated across the task. However, the Department is willing to consider proposals for combined task leads as long as there are clear lines of authority and only one individual responsible for reporting to the Director and/or Deputy Director and only one individual responsible for communicating with the Department. In this proposal, offerors shall discuss how they would ensure work is integrated across tasks.

Comment: Are each of the 10 named personnel required to be an author of the proposal?

Response: No, but note that the instructions require each offeror to identify the author of each section.

Comment: Page 67 of the RFI contains the following statement in the Instructions to Offerors: “In addition, this appendix shall contain letters of commitment from all proposed subcontractors, consultants, professional staff not currently employed by the offeror, or collaborators.” Is this referring to collaborators as in partners/research alliances? Or is this referring to unpaid collaborators for the work?

Response: This clause refers to subcontractors, teaming partners, consultants, and individuals who have pending employment letters from the offeror. It does not refer to unpaid collaborators.

Websites

Comment: Can we link to the REL website from our individual corporate website? How much information can be provided (and in what format) on a corporate website to not be considered overlap/an individual project website?

Response: Any information on a corporate site is outside of the scope of this project and will not be paid for using REL funds or approved by the IES. The REL may not direct people to their corporate site while conducting any activities paid for using REL funds.

PWS Task 1.2:

Comment: Task 1.2 contains the following requirement: "The Director or Deputy Director shall communicate with the COR before and after each meeting or conference call to discuss main points and follow-up as needed." Is the expectation that this communication will take place via email?

Response: Yes. Meeting agendas and summary notes are typically provided by the contractor via email.

PWS Task 2.1:

Comment: Recommend striking "plan joint activities that include other REL regions beyond the Governing Board’s region"-- Governing Board members are selected based on their knowledge of their state and region, not specific activities beyond the region or national issues.

Response: We decline to make this change. While we agree that the primary focus for these activities is regional in nature, there should also be a national focus.

Comment: Will IES provide a template for the Governing Board members to use to provide the annual, independent assessment? This places a significant amount of work on the part of the Governing Board.

Response: The PWS has been revised to provide additional information on the role of the Governing Board and the assessment that the Board is expected to perform.

PWS Task 2.3:

Comment: Please define “as appropriate” in the bullet that notes “[i]ndependent peer review is conducted as appropriate.”

Response: Legislation requires that all REL research be peer reviewed. IES or the REL may decide that peer review is also appropriate or useful for other REL activities, depending on the content or context.

Comment: Clarification: Are Technical Working Groups also needed for the TCTS activity?

Response: It depends on the nature of the TCTS activity. In consultation with IES, the REL contractors will need to exercise good professional judgment when determining whether a TWG will add value to the planning or implementation of a TCTS activity. Generally speaking, quality assurance for TCTS activities will not include a TWG.

PWS Task 2.4:

Comment: A due date of 10 working days after the end of each month is more reasonable to allow sufficient time for content development and internal review. From a finance standpoint, the books are not closed until four to five working days after the end of the month.

Response: The six-day deadline for Monthly Reports is required in order for IES to comply with mandatory IT reporting requirements that occur by the middle of each month. We expect contractors to maintain active estimates of expenses as the project progresses, and we believe that this data should be available prior to the formal closure of each monthly accounting period for a particular business.

Comment: Will the department consider quarterly reporting rather than monthly reporting?

Response: The monthly reporting requirement is not within the control of IES. Quarterly reports do not provide IES with sufficient data to satisfy federal IT reporting requirements.

PWS Task 2.7:

Comment: Can you provide more information about what the focus of the site visits will be? Given your response to the evaluation question on page 19 stating that the evaluators will be determined in Year 2, who will conduct the site visits?

Response: All details related to the evaluation of the REL Program will be determined when the performance work statement is drafted, the procurement is competed, and the contract is awarded to the successful offeror. REL contractors are required to cooperate with the evaluation and are asked to plan for site visitors as a placeholder until the specific details are available.

PWS Task 3.3

Comment: In proposals, should contractors suggest areas/topics where they are particularly well suited to coordinate activities across RELs?

Response: Yes, and the instructions indicate that each offeror should indicate up to three.

Comment: How will IES assign topics to RELs if multiple RELs sign up for the same topic?

Response: IES will determine the final topic selections.

PWS Task 4.2:

Comment: Does the Department intend to no longer refer to these types of activities as “Bridge Events”?

Response: Bridge events are intended to convey that they are opportunities to “bridge” research and practice. Activities that fit this definition may still be called “Bridge Events.”

PWS Task 5.2:

Comment: The concept of researchers “coaching” practitioners to solve problems of practice may not hit the right note with practitioner partners. Consider alternative terms such as technical support and assistance.

Response: Technical assistance is a term that is broad and vague. We have replaced that term in the PWS with more specific information about technical assistance activities in the REL context. We recognize that framing and relationships are extremely important and note that contractors may use whatever term they prefer in the field with their partners.

PWS Task 5.6:

Comment: It would really be helpful if the RELs had some leeway and were allowed to remove items from the SFS form that did not pertain to a particular activity/product.

Response: The Department is open to getting REL feedback on the SFS survey and having RELs delete unnecessary questions, as long as IES is getting the information needed to appropriately gauge customer satisfaction. RELs may propose which subsection of questions they are going to use to their COR in the event proposal.

Comment: Is the expectation to administer the survey after each activity or after a sample of activities?

Response: RELs should communicate with their COR to determine the best way to appropriately gauge stakeholder feedback. While a sample may be appropriate for some types of activities, it may not be appropriate for others.

PWS Task 5.4:

Comment: Will the Department offer What Works Clearinghouse certification during the first months of the contract?

Response: Yes, the Department will announce this training in early 2017 for successful offerors.

PWS Task 6.5:

Comment: For RELs serving many more jurisdictions than other RELs, having the ability to do more Just in Time projects per year or to do up to 10 over the course of the contract in any given year, would provide more flexibility for the REL to be responsive to state and local emerging needs

Response: We disagree. There are ample opportunities within the scope of work to be responsive to emerging state and local needs.

Performance Work Statement 12.21.2015

I. Purpose and Authorizing Legislation

The U.S. Department of Education (the Department) intends to enter into a 60-month contract with a qualified entity to serve as the Regional Educational Laboratory (REL) for each of nine regions. See Appendix A for the names of the regions and their constituent states or jurisdictions. The REL will be part of a program of 10 Regional Educational Laboratories authorized under the Education Sciences Reform Act (ESRA) of 2002, Part D, Section 174, (20 U.S.C. 9564). The REL program is administered by the National Center for Education Evaluation and Regional Assistance (NCEE) at the Institute of Education Sciences (IES)[footnoteRef:2]. [2: http://www2.ed.gov/policy/rschstat/leg/PL107-279.pdf]

Authorizing legislation directs the RELs to carry out applied research and development, disseminate findings from scientifically-valid research, provide support for using research in education decision making, and coordinate their activities with other federally-funded technical assistance entities (such as the Comprehensive Centers and the Equity Assistance Centers). RELs serve state education agencies (SEAs), local education agencies (LEAs), state and local school boards, and schools funded by the Bureau of Indian Affairs, as appropriate. RELs are required to allocate no less than 25 percent of their resources to meeting the needs of rural areas, as defined by the U.S. Census Bureau. Each REL has a Governing Board that provides strategic direction, including helping to prioritize which regional needs should be the focus of the REL’s work. A Governing Board is comprised of the region’s chief State school officers or their designees and other regional education stakeholders. Because REL work is carried out under contract to the Department, REL products and services must be completed to the quality specifications required by IES.

II. Background

Over the past decade, the REL program has contributed to a significant cultural change that continues to gain momentum in education – namely, the increasing focus on basing education decisions on high-quality research evidence. Evidence-based decision making in education has bipartisan support at the national level. Since 2002, the Department has made significant investments in an infrastructure to support building and using research evidence in education. In addition to offering the REL program, the Department has provided substantial support to states to develop longitudinal K-12 data systems; supported pre-doctoral and post-doctoral training of researchers capable of conducting rigorous studies; awarded grants for research ranging from exploratory and correlational studies to large-scale studies of effectiveness; conducted rigorous evaluations of federal programs; and established the What Works ClearinghouseTM (WWC) to search for, review, and summarize findings from studies of effectiveness in education. With the National Science Foundation, the Department’s Institute of Education Sciences articulated a set of Common Guidelines for Research and Development in Education (IES & NSF, 2013). Since 2008, the Department has added evidence-related requirements to a growing number of discretionary grant programs, of which the Investing in Innovation program is the most prominent example.

During this time, there arguably was more progress in training highly-skilled education researchers and producing high-quality research than in plainly communicating research results and providing support to policymakers and practitioners[footnoteRef:3] to engage with research and evidence, either as research partners or research consumers. The 2012-2017 REL program sought to address the disconnect between researchers and practitioners through researcher-practitioner partnerships known as research alliances. Research alliances were defined[footnoteRef:4] in the Performance Work Statement (PWS) as groups of practitioners, policymakers, and researchers who work together over time to use data and research to better understand and address a particular education concern. REL research alliances typically involved cross-state membership, groups of districts within a single state, or both state education agencies and several districts. The premise of this approach was that partnerships of researchers, practitioners, and policymakers would result in high quality research studies and products that answered pressing practical questions, communicated findings in plain-spoken but technically precise ways, were engaging and actionable for practitioners, and — most important — built the capacity in states and districts to apply research results. [3: Throughout this PWS, we refer to policymakers and practitioners. Our working definition of policymaker is an individual who has the authority to make policy choices for an education system. This definition, then, reasonably includes members of state or local boards of education, state education officials, and superintendents. Our working definition of practitioner is an individual whose responsibilities primarily are to carry out education practice, rather than to set broad policies. Practitioners, then, would include school principals, instructional coaches, and teachers. Our purpose in providing these working definitions is to indicate the various types of partners and audiences for REL activities. ] [4: See: https://www.fbo.gov/index?s=opportunity&mode=form&id=1918ddb4c35fe795cb256350128f1365&tab=documents&tabmode=list]

The Department intends for the 2017-2022 REL program to build on and extend its recent history of high-quality research, genuine partnership, and effective communication. Therefore, the 2017-2022 RELs are tasked with conducting rigorous research, working in partnership with state and district staff, and effectively communicating about research. In addition, the Department expects the RELs to apply these capabilities to high-leverage education problems in their regions. High-leverage problems are those that (1) if addressed could result in substantial improvements in education outcomes for many students or for key subgroups of students; (2) are priorities for regional policymakers, particularly at the state level; and (3) require research or research-related support to address well. Focusing on high-leverage problems increases the likelihood that REL support ultimately will contribute to improved student outcomes.

RELs conduct three types of activities

ESRA charges RELs with three main activities: (1) applied research; (2) dissemination of scientifically-valid research; and (3) technical assistance related to application and use of scientifically-valid research. While there are not always bright dividing lines between these three activities (for example, a research activity may also include support or training for staff at state or local education agencies), these categories are useful as general descriptors of activities that RELs may undertake to inform and increase the capacity of policymakers and practitioners in their regions to access, conduct, interpret, and apply research.

Applied research

ESRA describes applied research as specifically directed to the advancement of practice in education. The law implicitly contrasts applied research with basic research, which does not necessarily have immediate or obvious implications for practice. The Department envisions the applied research of the RELs as use-inspired research[footnoteRef:5] -- that is, research that seeks to solve practical problems but also, to the extent possible, advances fundamental understandings of education problems and processes. Whether basic or applied, quality research is informed by prior empirical and theoretical work and seeks to add new knowledge to that body of work (IES & NSF, 2013).. Importantly, research is concerned with generalizing results to multiple contexts. Research may provideinvolve careful descriptions of patterns, examine examination of associations, or conduct rigorous empirical testing of models and hypotheses. In the education context, research addresses complex questions about teaching, learning, organization of schools and systems, education access, and education attainment. These characteristics separate research from other forms of information-gathering, such as compiling facts or statistics, documenting policies without analysis or scholarly interpretation, or generating and reporting on performance indicators. [5: Pasteur's Quadrant: Basic Science and Technological Innovation by Donald E. Stokes (1997). Washington, D.C.: The Brookings Institution ]

Dissemination of research findings

In ESRA, dissemination is characterized as the “communication and transfer” of research findings in ways that are “understandable, easily accessible, and usable, or adaptable for use in the improvement of educational practice.” RELs are to facilitate the flow of actionable, credible, up-to-date research evidence (and information, ideas, and approaches that are clearly based on credible research evidence) among researchers, practitioners, and policymakers. The Department emphasizes that it does not envision a “transmission” model in which the “experts” (i.e. researchers) convey information to non-researchers. Rather, REL dissemination is to facilitate the wide sharing of research evidence and research-related information to spark new ideas, introduce individuals to others working on similar issues, and reduce unnecessary duplication. In an age when information is ubiquitous and readily transmitted, the value-added of the RELs is that they are honest brokers and effective synthesizers of information. The Department’s goal is for the “REL brand” of dissemination to convey quality, objectivity, timeliness, and engaging opportunities to learn and connect with others.

The Department recognizes that dissemination activities, on their own, are unlikely to bring about deep and lasting improvements in education practice. This is because changes in organizational and instructional routines and practices typically require more intensive support than can be provided in presentations, workshops, webinars, or other relatively brief interactions (Christie et al 2007).. But dissemination can alert policymakers and practitioners to new findings and approaches and, ideally, to resources that can support deeper engagement in content. To help audiences to go beyond learning about research to applying research, RELs also are charged with providing more intensive, targeted training, coaching, and technical support to stakeholders, which is discussed below.

Training, coaching, and technical support for use of research

ESRA uses the term “technical assistance” to characterize a broad area of work that RELs conduct to support use of research evidence in practice and policy, including work to select and test solutions, improve classroom instruction and student learning, and plan and assess programs. In the 2012-2017 RELs, this area of work was called “analytic technical assistance” to distinguish it from assistance for policy and program development and implementation offered by other federal education technical assistance centers. In this contract, the Department is calling this work training, coaching, and technical support (TCTS) for research use. A wide variety of activities and partnership models are allowable within this general area of work, but all must be designed intentionally to increase the capacity of individuals and/or organizations, and, when appropriate, the education research sector (see discussion of capacity-building later in this introductory section). The content of activities conducted within this area of work must be carefully assessed against the research base. Further, unlike the “light touch” envisioned by dissemination, these activities imply more extensive, ongoing, or customized support to meet the needs of particular stakeholders. Some of these activities are services that do not result in tangible products for wide distribution. However, published products for a broad audience such as guides, toolkits, or videos that arise from technical support activities are allowable in this area of work.

Examples of training, coaching, and technical support to enable use of research include but are not limited to: in-person or virtual consultation or training on research design, data collection or analysis, or approaches for selecting or adapting research-based interventions to new contexts; in-depth training or workshops on instructional approaches supported by rigorous research, including preparation of materials for these trainings or workshops; informal reviews of studies against WWC standards[footnoteRef:6] (see subtask 5.4); acting as a thought partner when evaluating a program or initiative, and reference searches in response to questions submitted by stakeholders through the Ask-A-REL service (see subtask 5.5). [6: . The What Works Clearinghouse standards are available at http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v3_0_standards_handbook.pdf. ]

Supporting use of research evidence and building research evidence are defining features of REL work. Therefore, trainingTraining, coaching, and technical support activities that are not related to building should build capacity for creating, accessing, interpreting, or usingapplying research evidence are out. Though a key goal of scope for RELs. In particular, advising on program or research in education is promotion of evidence-informed policy development, alignment, or implementation aremaking and practice, it is not within the mission of the REL programto recommend, advocate for, or design policies.

Bridging Research and Practice in Education

The REL program serves as a bridge between the worlds of education research and education practice.

Though these worlds share a common desire to improve education outcomes, they often differ in their incentive structures, time constraints, priorities, and communication practices (Biesta 2007).. More than any other research-related program at the Department, the REL and its staff must be able to navigate comfortably between the two worlds and must understand the norms and requirements of both research and practice in education. Specific examples of how this requirement must be met include:

· Demonstrating rigor and relevance: The REL shall conduct research to the high standards expected by IES. However, the REL also shall be committed to co-creation and implementation of research agendas with its education partners and skilled at communicating research findings in an easily accessible manner to busy practitioners.

· Differentiating support based on need: The REL shall be able to provide its region with access to the most sophisticated research expertise. However, it also shall offer support on basic, introductory-level data and research topics when needed.

· Providing information in the short term, and developing evidence over the long term: The REL shall respond to immediate needs of practitioners and policymakers for products to inform decisions, usually within a short time horizon. However, REL researchers shall craft research projects so that they also make empirical and theoretical contributions to long-term, enduring issues in education research.

· Address local issues that can also benefit the nation: The REL shall respond to regional needs for research, dissemination, and technical support as articulated by its education stakeholders. However, as much as possible, it also shall design its activities to benefit practitioners and policymakers throughout the entire country. To the extent possible, the REL shall craft its work to address the research needs of local partners while also informing broader scholarly and policy debates.

Clientele The primary clients for the REL are the state and local education agencies, state and local boards of education, and schools funded by the Bureau of Indian Affairs in its region. However, to maximize the benefit of REL resources, the Department encourages RELs to consider how secondary clients can benefit from work initially designed to meet the needs of a primary client. Secondary clients include other education stakeholders in a region, such as parents, students, and the general public; a national audience of other state and local education agencies or boards of education; and education researchers throughout the country.

III. TOPICAL PRIORITIES

Priorities for the REL are described throughout this PWS, especially under Tasks 3, 4, and 5 where especially encouraged activities are listed. The Department emphasizes that the REL is to work on high-leverage issues in its region, as defined in Section II. However, to the extent possible, the Department also encourages the REL and its region to address the following four topics: (1) early childhood education and school readiness; (2) college and career readiness; (3) professional development and teacher preparation to deliver effective instruction, informed by the most credible, up-to-date research; and (4) increasing education access and attainment in high needs communities, schools, and subgroups of students. This set of priorities is offered as general encouragement. To the extent possible, the REL and its region should address these topics if they meet the regional needs, are high-leverage topics, and meet the other qualifications described throughout this performance work statement.

IV. EXPECTATIONS FOR IMPLEMENTATION OF THE REL

1. The Department expects the REL to make efficient and effective use of its resources for greatest impact and broad benefit.

First and foremost, the REL serves its region by learning about, prioritizing, and addressing regional needs. The REL’s Governing Board, which includes but is not limited to the State chief or designee from each state in the region, helps the REL allocate its resources to various activities in the region and ensure equitable distribution of work across the jurisdictions. Governing Board members also can help the REL make connections with other regional and national entities that can co-sponsor, extend, or amplify the REL’s work in order to reach new audiences within the region. However, as directed in authorizing legislation, the REL also must seek to address national priorities (see Section III) and to benefit education policymakers and practitioners across the entire country.

Expectation: The REL shall thoughtfully and strategically use its resources to generate the greatest good for the greatest number of individuals and entities, both in the region and across the country. The first way that RELs shall accomplish this is through identifying high-leverage education problems on which to work, drawing on careful assessment of regional needs and opportunities and on the guidance of their Governing Boards. High-leverage problems are those that (1) if addressed could result in substantial improvements in education outcomes for many students or for key subgroups of students; (2) are priorities for regional policymakers (particularly at the state level); and (3) require research or research-related support to address well.

A second way that the…