PWS - Task Order 34.pdf

Version Date: January 20, 2022

Performance Work Statement Period of Performance: July 1, 2022 – June 30, 2023 Bridge Contract: Task 34: CRDC Research and Evaluation Support

The Civil Rights Data Collection (CRDC) is a mandatory data collection that provides vital data on education equity for the U.S. Department of Education’s (ED or the Department) Office for Civil Rights (OCR). The CRDC is authorized under the statutes and regulations implementing Title VI of the Civil Rights Act of 1964, Title IX of the Education Amendments of 1972, Section 504 of the Rehabilitation Act of 1973, and under the Department of Education Organization Act (20 U.S.C.

3413). The regulations implementing these provisions can be found at 34 CFR 100.6(b); 34 CFR 106.71; and 34 CFR 104.61.

The CRDC is a longstanding and critical aspect of the overall enforcement and monitoring strategy by OCR to ensure that recipients of the Department’s federal financial assistance do not discriminate on the basis of race, color, national origin, sex, and disability. OCR relies on the CRDC data received from public school districts as it investigates complaints alleging discrimination, determines whether the federal civil rights laws it enforces have been violated, initiates proactive compliance reviews to focus on particularly acute or nationwide civil rights compliance problems, and provides policy guidance and technical assistance to educational institutions, parents, students, and others.

The data are also used by the Department’s National Center for Education Statistics (NCES) within the Institute for Education Sciences (IES) and other offices, as well as policy makers and researchers outside of ED. The CRDC provides a measure of the equity health of schools across the nation by collecting wide-ranging data on educational access and opportunity. The CRDC collects data on early childhood education, college and career readiness, characteristics and demographics of schools, discipline, and resource equity.

The CRDC is administered every two years to public school districts or Local Education Agency (District or LEA). The survey began in 1968, traditionally as a sample survey of approximately 6,000 school districts covering each of the 50 states, the District of Columbia, and Puerto Rico.

Beginning with the 2011-12 collection, the CRDC administrations have been a universe collection from every public school district and school. The CRDC includes traditional public schools, alternative schools, state-operated facilities, and juvenile justice facilities.

To meet the purpose and intended uses of the data, the CRDC collects information about students in public schools, including enrollment, educational programs and services, disaggregated by race/ethnicity, sex, limited English proficiency and disability. Additionally, the location of where the students receive instruction is reported so that the data OCR collects reflects an accurate picture of students’ access to educational opportunities at the site where they spend the majority of their school day.

Each LEA submits a short survey about the LEA as well as a longer survey about each school in the LEA. Since 2004, the CRDC has been conducted primarily online. The web-based survey is filled out by representatives of LEAs who use state, LEA, and school information systems and other resources to respond to questions. The LEA is the respondent and the responsible entity for certifying to the accuracy of the data submission. Beginning with the 2013-14 CRDC administration, a group of State Education Agencies (SEAs) pre-populated LEA survey forms with data already collected by the SEA for state reporting purposes. LEAs in these states are able to review and amend as needed the SEA prepopulated data before certifying their CRDC. The two recent CRDC including data from the 2015-16 and the 2017-18 school years.

II. Purpose

The primary purpose of this task is to support data quality review tasks, research, and evaluation of the CRDC. This task also includes the development and creation of CRDC-related research and/or data products that will be published by the NCES. The work in this task will improve the overall quality, efficiency, and usability of the CRDC. Contractors are expected to collaborate with NCES, the OCR, other ED program offices, and other contractors.

The secondary purpose of this task is to contribute to and enhance NCES administrative and other survey collection programs. This work will include projects to boost NCES and the public’s access and use of CRDC files. The contractor is expected to develop data, research, and evaluation projects, and disseminate papers and blogs for external and internal stakeholders. The CRDC includes data from as early as 1968, and the contractor should expect to use more recent CRDC data files and documentation.

The data to be used for the following tasks may include previous CRDC collections (including and prior to SY 2015-16) and the current CRDC collection.

III. Performance Tasks

Aside from overall project management specified in Task 1 and transition activities specified in Optional Task 5, ED expects the following overall effort for this work:

TASK LEVEL OF EFFORT

Task 2: CRDC-NCES Research and Technical Products Task 3: Data Quality Review Task 4: Technical White Paper 100% (excluding optional task)

**The level of effort for these tasks may change if ED elects to execute the optional task.

TASK 1: MANAGEMENT AND COLLABORATION

Coordination with NCES and oversight of the project will be critical to its continued success. This task covers project management and administrative aspects of that effort.

Sub-Tasks

1.1 Kick-off Meeting: Within one week of project award, the contractor’s key staff will meet with NCES and OCR to discuss the project goals, project timeline, and to clarify any aspects of the work. The contractor should plan to review the overall contract tasks, identify information from NCES that would be useful for the project, identify potential problems and solutions, and discuss areas for concern related to the proposed project staffing plan and other management requirements.

Deliverables:

The contractor will submit an agenda three business days before the meeting and provide minutes from the meeting within five business days.

1.2 Bi-Weekly Status Meetings: The Project Director and the appropriate staff identified by

NCES will meet bi-weekly to discuss the project status. Typically, these meetings are not longer than one hour.

Deliverables:

The contractor should provide an agenda two business days before each scheduled meeting and meeting minutes within two business days following the meeting.

TASK 2: CRDC-NCES RESEARCH AND TECHNICAL PROJECTS

The purpose of this task is to bolster the public’s use of CRDC data. The contractor will collaborate with NCES and OCR staff, ED Offices and other contractors to develop data, research, and evaluation products accessible to ED and the external public to improve the CRDC quality, production efficiency, statistical methodology, or data coverage.

Sub-Tasks

2.1 CRDC Related Entries for NCES Blog: The goal of this task to create and publish at least three blogs to make the public aware of the CRDC and its usability in conjunction with other NCES resources.

2.2 CRDC Data Story: NCES is interested in using CRDC data as well as other NCES data to explore and study issues related to student success and access. The contractor should expect to collaborate with NCES and OCR staff along with other contractors to develop and execute a research study focused on data collected within CRDC and present results of the analysis through an interactive, web-based data story (i.e., data visualization).

2.3 CRDC Archival Project: Under the previous periods of performance in this Task Order, historical CRDC data files and documentation were successfully accessed and organized.

NCES wants to continue this work with the goal of raising public awareness and increasing access and use of the released CRDC Archival Project. The contractor will collaborate with NCES and OCR staff and other contractors to perform the following subtasks:

2.3.1 Research and Analysis Output: The goal of this task is to disseminate information about historical CRDC data via at least one NCES release in attractive and easily readable formats for the public. Potential research projects could include:

o A comprehensive data dictionary and codebook. Document should be easily accessible for users to access in an electronic format.

o A detailed longitudinal analysis of CRDC (1968 – 2007) o A report using the CRDC archival data resulting in a public facing document or tool

2.1 – At least three well-written blog posts that adhere to NCES publishing guidelines.

2.2 - An interactive, web-based data story that adheres to NCES publishing guidelines.

2.3.1 – A visual product that adheres to NCES publishing guidelines.

TASK 3: DATA QUALITY REVIEW

Under this task, the contractor will engage in evaluation and data quality review activities to assist OCR in improving the data quality of future CRDC data collections. The contractor may use past or current CRDC collections to make recommendations and inform strategies.

Sub-Tasks

Specific projects will include the following:

3.1 Machine Learning to Support Data Quality: For this task, the contractor will continue the work to examine the use of machine learning and data science methods for outlier and missing data detection in administrative data collections. The contractor will collaborate with other contractors to determine best strategies for integrating advanced analytic methods into the CRDC data submission, post-submission, and corrections processes. The contractor will use older and the current CRDC collections to accomplish the task goals.

3.2 Data Quality Work: The contractor will collaborate with NCES to provide support to OCR and OCR’s contractors on important data quality tasks including the development and editing of business rules, the review of data, the assessment of risks. Some of these tasks may include an exploration of a risk matrix to assess potential threats to data quality and an evaluation of the alternative action plans (or quick plans). The contractor should expect to collaborate with NCES, OCR, other ED offices, and other contractors to develop work plans and to identify specific data elements.

Deliverables

3.1 An evaluation report examining the use and effectiveness of machine learning and data science methods for outlier and missing data detection. The report should also include best strategies for integrating advanced analytic methods into the CRDC data submission, post-submission, and corrections processes.

3.2 An evaluation report on the utilization of the quick plans. Potential areas of exploration for this evaluation may include the following: how OCR should use information in the quick plans, misreporting of zeros, policies that prevent LEAs from reporting data to the CRDC, etc.

TASK 4: TECHNICAL WHITE PAPER (INTERNAL)

For this task, the contractor will collaborate with ED staff, NCES, OCR and other ED Offices and contractors to develop a technical proposal, for use within ED, on data quality recommendations arising out of the CRDC data collection cycle and work done on Task 3, specifically on how to operationalize risks or threats to CRDC data quality. Task 3 is focused on performing a review of the most recent collected data and the process in use for the collection cycle. The intent of Task 4 is to gather recommendations from the contract team on ways to 1) operationalize risks, 2) reduce threats and 3) improve the overall data quality process and automated data quality routines used for CRDC after the 2020-21 cycle. The technical white paper developed under Task 4 is not for publication or use outside of ED. The paper should synthesize lessons the contract team learned throughout the CRDC collection cycle (e.g., pre-collection, collection, and post-collection) and provide a list of recommendations about how the data collection process can help OCR identify risks and other potential threats to enhance data quality for CRDC and improve the CRDC collection and review process. The purpose of Task 4 is to have the contractor organize a set of recommendations for ED to consider that will build upon what was learned by the contract team in performing Task 3.

The Technical White Paper should be no more than 10-15 pages, not including any supporting appendices. The contractor will work with ED to develop plans and timelines for the white paper at a time when deliverables on Task 3 are nearing completion.

4a - Preliminary drafts of agreed upon paper.

4b – Final draft of paper, incorporating revisions based on ED input.

TASK 5 (OPTIONAL): TRANSITION ACTIVITIES

The Government may decide to implement a transition period if funding permits. The transition period, if implemented, would not affect the contract periodicity specified above. If a transition period is necessary, the Period of Performance will be 30 days with dates to be agreed upon by the government and the contractor. During this transition period, the NCES and the contractor shall identify tasks and agree on the scope of work. Transition hours shall also be budgeted to efficiently transfer necessary files to the Government.

IV. Period of Performance

The period of performance for this task order is from July 1, 2022 to June 30, 2023.

V. Schedule of Deliverables This schedule assumes a period of performance for this task order of July 1, 2022 to June 30, 2023.

During the kick-off meeting, the contractor, task leader, COR and other ED-NCES project managers will establish the project timeline with keys dates for deliverables.

Task 1: Management and Collaboration

1.1 – Kick-off meeting

1.2 – Bi-weekly status meetings

Agenda Meeting minutes

Agenda 3 days before meeting; Minutes within 5 business days after the meeting

Biweekly – draft Agenda provided 2 business days before the meeting; final minutes provided 2 business days after i

Task 2: CRDC-NCES Research and Technical Projects

2.1 – 3 CRDC

Blogs

Blog post Drafted Blog posts provided 1 week after assignment; final Blog post 1 week after comments; contractor may invoice after final approval of blog post

2.2 – Web-based, interactive, data story

Final data story Draft data story, 2 weeks after assignment.

Final story 2 weeks after feedback.

Contractor may invoice after final data story

2.3.1 – CRDC

Archival Project Research output

A visual product (e.g., data dictionary/codebook) that adheres to NCES publishing guidelines

The contractor will develop a work schedule during the kick-off meeting.

Contractor may invoice for this on a monthly basis after acceptance of each deliverable

Task 3: Data Quality Review

3.1 – Machine Learning to Support Data Quality

An evaluation report examining the use and effectiveness of machine learning and data science methods for outlier and missing data detection. The report should also include best strategies for integrating advanced analytic methods into the CRDC data submission, post-submission, and corrections processes.

Draft report within 2 months of assignment;

final report within one month after receiving feedback on the draft report.

Contractor may invoice ½ for draft report and remaining ½ after final t

3.2 – Data Quality Work

An evaluation report on the utilization of the quick plans.

Contractor may invoice ½ after acceptance of draft memo and remaining ½ after final

Task 4: Technical White Paper (Internal)

4.1a - Preliminary draft of agreed upon paper.

A draft proposal for use within ED on data quality recommendations arising out of the CRDC data collection cycle and work done and lessons learned on Task 3, specifically on how to operationalize risks or threats to CRDC data quality.

The contractor will develop a work schedule during the kick-off meeting.

Contractor may invoice for this on a monthly basis after acceptance of each deliverable

4.1b – Final draft of paper, incorporating revisions based on

Final paper reflecting result of review and discussion period with ED on initial draft.

Final paper within 2 weeks of finalizing review and discussion period with ED.

Task 5 (Optional): Transition Activities

5 -Report on final transition activities

NCES and the contractor shall identify tasks and agree on the scope of work

Within 30 days of government exercise of the option.