PWS for TRIO Data Analysis_SR_1.28.21.docx

OFFICE OF POSTSECONDARY EDUCATION

TRIO DATA ANALYSIS

Performance Work Statement

Version Date: 01.28.21

Background General background: The mission of the Student Service (SS) Area of the Office of Postsecondary Education (OPE) is to enhance educational opportunities for low-income individuals, first-generation college students, and individuals with disabilities. SS fulfills this mission by administering grants that fund institutions and organizations (grantees) to implement programs for this purpose. SS provides technical assistance to grantees and potential applicants; provides statistical information to internal users for decision-making and program improvement; and develops statistical reports on the programs for the external community.

Among the grant programs administered by SS are the following Federal TRIO Programs authorized in Title IV, Part A, Subpart 2, Chapter I of the Higher Education Act of 1965, as amended: Upward Bound (UB), Upward Bound Math-Science (UBMS), Veterans Upward Bound (VUB), Student Support Services (SSS), Talent Search (TS), Educational Opportunity Centers (EOC), and Ronald E. McNair Postbaccalaureate Achievement (McNair). These programs represent the largest set of discretionary grant programs in the Department of Education (the Department) with close to 3,000 grant awards made annually.

UB, TS, and SSS—the original TRIO programs--were established in the 1960s to aid disadvantaged students in preparing for, and succeeding in, postsecondary education. EOC and VUB were subsequently established to increase postsecondary opportunities for adults and veterans. McNair was established in 1986 and implemented in 1989 to prepare students from disadvantaged backgrounds for doctoral studies through involvement in research and other scholarly activities. The most recent TRIO program, UBMS, was created in 1990 to address specifically the need for additional math and science instruction at the secondary level and to encourage disadvantaged students to enter postsecondary education in those fields.

Annual Performance Reports (APR) System: The Department collects annual performance reports for each of the TRIO outreach and support programs. The Upward Bound and Upward Bound Math-Science programs APR submissions are managed through a single, unified collection (the “UB/UBMS” collection), whereas each of the five other programs have discrete collections. These APRs serve to provide the data necessary to respond to the requirements of the Government Performance and Results Act (GPRA), to annually assess a grantee’s progress in meeting its approved objectives, and to award prior experience (PE) points in accordance with the authorizing statute and programs regulations. For both the TS and EOC programs, grantees must submit project-level data through a web-based form. TS and EOC projects aggregate information about project participants, but do not submit participant (or individual) level records. For the SSS, McNair, UB/UBMS, and VUB programs, grantees must submit annually a file that includes data on each participant served by the project during the reporting year as well as updated information on prior participants.

Copies of the OMB-approved performance report forms for these programs, along with instructions for using the Web sites, are available at the following Web addresses:

http://www2.ed.gov/programs/triostudsupp/report.html http://www2.ed.gov/programs/triomcnair/report.html http://www2.ed.gov/programs/trioeoc/report.html http://www2.ed.gov/programs/triotalent/report.html http://www2.ed.gov/programs/trioupbound/report.html http://www2.ed.gov/programs/triomathsci/report.html http://www2.ed.gov/programs/triovub/report.html

Data reporting in TRIO: SS collects APR data through a web-based information technology system. The Department employs a different Contractor (the “Web Hosting/Data Processing Contractor,” more briefly referred to as “the Data Collection Contractor”) to support this APR collection system. After the annual APR collection period each fall, the Data Collection Contractor is responsible for securely compiling and transferring APR data to the Analysis Contractor. The Analysis Contractor then uses the data to complete its tasks and deliverables for this task order.

For the UB, UBMS, VUB, SSS, and McNair programs, the Department collects individual-level participant data from grant projects during the annual APR submission. Grant projects update individual records each year that they submit an APR. The Analysis Contractor maintains a longitudinal data file which provides the capacity to track individuals over time through multiple years of data collections.

APR data submitted in the future will be merged with the existing applicable longitudinal file. Table 1 shows the number of grants awarded under each of the TRIO programs in fiscal year 2019 (program year 2018-19). Table 1 also indicates which TRIO programs work with high school students and adults prior to their enrollment in college, as opposed to the programs that serve students enrolled in college programs.

Table 1: TRIO Program, by Number of Grant Projects (2018-19 Academic Year) and Participant Academic Level

The Higher Education Opportunity Act of 2008 (HEOA): Following enactment of the HEOA, which amended the Higher Education Act of 1965, the Department conducted a negotiated rulemaking process to develop regulations to implement the changes required by the HEOA. The final program regulations are available in parts 642-646 at https://www.ecfr.gov/cgi-bin/text-idx?SID=434afa9981af62690569a7dd214c55dc&mc=true&tpl=/ecfrbrowse/Title34/34chapterVI.tpl.

Requirements The Department requires Contractor support in conducting analysis of grantees’ data, as presented in their APRs. The TRIO Analysis Contractor will provide a secure File Transfer Protocol (FTP) site so that the Department’s TRIO APR Web Hosting/Data Processing Contractor can safely transfer the APR data. The Department’s Contracting Officer’s Representative (COR) who manages the TRIO Analysis contract will facilitate the transfer of the APR data. The Analysis Contractor shall:

· analyze the quality and completeness of the APR data;

· merge/append the annual APR data to the longitudinal data files for the SSS, McNair, UB/UBMS, and VUB programs;

· update the database backbone of an electronic codebook/data element dictionary for the longitudinal files;

· analyze grantee and program outcomes in accordance with GPRA requirements;

· leverage APR and other data to support research efforts;

· produce data tables and reports for dissemination; and

· maintain existing geographic maps of interest to TRIO program stakeholders:

TRIO Story Map TRIO Explorer Map Discover Target School Map Talent Search Target Schools Upward Bound Target Schools (not yet posted) TRIO Training Map (not yet posted)

Tasks The data analysis tasks will entail work and deliverables that apply to each of the programs, with some variations and exceptions. This PWS is therefore organized by task, rather than by individual program. Where specific details need to be stated for individual programs, those details are provided after the overarching description of the task.

The Schedule of Deliverables (in Part E) provides estimated due dates for the tasks described in this PWS for the Base Period of the contract. Since the schedule is dependent on the contract award date and receipt of the 2020-21 APR databases from the Web hosting/data processing Contractor, some adjustments to the schedule may be required. Such schedule adjustments may also be needed if the Office of Management and Budget (OMB) approval of the APRs is delayed, if there are delays in developing and testing the APR Web applications, and/or if the Department changes the APR due date for grantees.

Task 1: Project Management

The Contractor shall provide a variety of project management support functions for the tasks described in this PWS to ensure that all tasks are appropriately staffed, and deadlines are met.

Subtask 1.1: Kick-off Meeting

Within ten business days after the award of the contract, the Contractor shall convene a kick-off meeting with SS staff to introduce team members, initiate the tasks, clarify expectations, establish communication protocols and revise timelines and deliverables. The Contractor shall provide an agenda three business days prior to the meeting. Within five business days following the meeting, the Contractor shall submit meeting minutes. The Contractor shall submit an updated management/work plan and a revised Schedule of Deliverables with firm dates for COR approval within ten business days after the kick-off meeting.

Subtask 1.2: Task Planning, Status Reports, and Communications

Task Planning Meetings/Conference Calls. Prior to beginning work on each new task/subtask for each program (e.g., analysis of quality of TS APR data, UB GPRA report, etc.), the Contractor and Departmental staff shall meet virtually to review the task requirements, clarify expectations, and respond to questions. Because of the differences in the APR data and performance measures for each of the TRIO programs, the Contractor shall meet virtually in the Base Period with Departmental staff prior to beginning work on the deliverable for each program. For example, for Task 3.1 (APR Data Quality Memoranda).

The Department requires a high level of accuracy in TRIO reports and products; in certain “high stakes” deliverables, accuracy is paramount. Therefore, the Contractor shall use these task meetings, as well as the ongoing communications described below, to bring to the COR’s attention any issues that need consideration and resolution to ensure accuracy and completeness. The Contractor shall prepare for and administratively support these task planning meetings and provide a detailed list of issues and/or questions three business days prior to each meeting. Within five business days after each meeting, the Contractor shall prepare and distribute meeting minutes to the designated Departmental manager.

Monthly Status Reports. The Contractor shall submit a Monthly Status Report within ten business days following the end of the calendar month. The monthly report shall include a detailed summary of accomplishments; a table showing each deliverable’s number, title, due date, and delivery date; the status of outstanding deliverables; and a description of problems encountered and the resolution of the problems.

Ongoing communications. In addition to the above requirements, the Contractor must be available to Departmental staff by telephone and e-mail to discuss current issues, concerns, and problems related to contracted tasks and deliverables. Because of the dynamic nature of the work, many times these contacts cannot be scheduled in advance. Further, the Contractor must notify Departmental staff within ten business days of any problems that may impact on the Contractor’s ability to meet the agreed upon dates for task deliverables. This includes problems with data availability and quality and changes in Contractor staff or the time commitment of staff assigned to this contract. Should the communications involve substantive issues and decisions, the Contractor shall prepare and submit minutes within three business days.

The monthly project management needs may vary throughout the contract year because of the level of effort needed to manage the tasks during peak time and non-peak times.

Deliverables:

1.1.1 Kick-off Meeting Minutes

1.1.2 Revised management plan and revised schedules

1.2.1 Minutes of Task Planning Meetings

1.2.2 Monthly Management Reports

Optional Task 2: Transition In/Transition Out Subtask 2.1: Transition In -- Review of Data Files from 2017-21 Contract

The COR shall give the Contractor all files, programs, relevant deliverables, and documentation of work and products completed under the 2017–21 contract within 20 business days of contract award. Furthermore, within 20 business days of the contract award the Department will give the Contractor access to prior years’ APR data and documentation, including the student record files. The Analysis Contractor can expect to receive the customized electronic codebook (ECB) files, programs used to generate the codebooks and the Access database backbone that the ECB draws from; data processing files used; statistical code used in merging other data files; data analysis programs; historical files on APR record layout; and deliverables produced under the 2017–21 contract that will be relevant to the new contract. The Contractor must be able to work with data in standard statistical program packages, such as SPSS or SAS. Any data files submitted to ED in statistical software (for example, in Task 4) must be provided in a format ready for use on any major statistical platform that ED requires (e.g., SAS, SPSS, STATA, etc.). Currently, ED staff in TRIO only have access to SPSS. So, while the Contractor may work in any package of their choosing, the Contractor must have access to a tool such as StatTransfer, to deliver data in a format with which staff in TRIO can work.

Subtask 2.2: Transition Out – Transfer of TRIO Data Resources and Documentation

This subtask is not required in the Base Period.

Deliverable:

2.1.1 Report of results of review of data files and documentation Task 3: Analysis of Quality of APR Data

Background: The TRIO Analysis Contractor will receive six discrete databases, each of which will constitute the completed data collection for a TRIO program. (Refer to Table 1 on page 2 for list of databases.) In the Base Period (2021–22), the Analysis Contractor will assess the quality of data from grantees’ 2020–21 reporting period (Subtask 3.2). Grant projects will submit this data to the Department between November 2021 and March 2022. In order to prepare for this assessment, the Contractor will develop metrics using data collected in the prior year (Subtask 3.1).

Subtask 3.1 Data Reliability Indices

The TRIO Analysis Contractor shall deliver a report that provides documented code prepared to create two individual-level data reliability indices for each of the four programs (eight total indices) with longitudinal data files (see Task 4). Both indices will provide a global summary metric at the individual (i.e. ‘student’) level regarding the consistency of reporting. The report, in addition to providing documented code, will provide a clear and concise summary description of each index, including sufficient guidance as how the index should be interpreted.

For each of the four programs, one index will indicate the ‘across-field, within year’ reliability of reporting, with a focus on the extent of inconsistencies within a single student record regarding information in two or more fields. An individual record without any within-year inconsistencies across fields would receive a perfect score on this index. The greater the extent of within-year inconsistencies across fields, the lower the score on this index.

The second index will indicate the ‘across-year, within-field’ reliability of reporting, with a focus on the extent of inconsistencies within a single record between reporting in one field (or one set of fields) over time. An individual record without any within-field inconsistencies across years would receive a perfect score on this index. Additionally, a new individual without any prior records would also have a perfect score on this index. The greater the extent of within-field inconsistencies across-years, the lower the score on this index.

The indices should be easy to read and interpret. They should be useful to identify sharp distinctions between records that are moderately problematic and ones that are very problematic. The indices should be useful for aggregation up to the grant project level, such that the reliability of grant project reporting can be measured in terms of reliability across the individual records that the grant project has submitted (weighted for the number of records that the grant project submits). In this way, the indices should be useful to identify specific grant projects that have problematic reporting. Finally, the indices should be useful to identify patterns of reporting inconsistencies. It should be possible to use the indices to determine what specific set of inconsistencies are most frequent.

In order to create these two indices, the Contractor shall use the Annual Performance Reports for the 2019-20 project year and the longitudinal files that cover through the 2018-19 project year.

Subtask 3.2 Data Quality Memorandum

The Contractor shall prepare a Data Quality Memorandum (DQM) for each grant program. Each program’s DQM will consist of a narrative report supported by tables and figures that provide useful information regarding reliability of reporting. The information shall be presented in a clear and concise manner. It shall provide global metrics of reporting reliability at the program level, and display, clearly and concisely, distributions across metrics at the project level. For the four programs that submit a student record file as part of their APR submission, the global reliability metrics will be the two indices developed in Subtask 3.1. Global metrics of reporting reliability shall be compared with prior year metrics.

The data quality memorandum shall include a section that addresses patterns of reporting inconsistencies across fields, and over time. The presentation of this information shall be identified in a clear and concise manner. Grant projects that have high volumes of reporting reliability issues shall be identified, and their reporting reliability over time shall be charted. Finally, the DQM shall respond to specific questions that TRIO staff members may raise about the data.

The data quality memorandum shall include a section that addresses data validations and reporting of out-of-range values. Both data validations and reporting of out-of-range values refer to reporting constraints that the data collection Contractor is required to build into the data collection system so as to prevent grant projects from submitting invalid APR data. Data validations refer to requirements that should prevent grant projects from reporting inconsistencies across fields of data; reporting out-of-range values refer to requirements that prevent reporting of data that has been defined as not within the range (or set) of allowable options. The section of the DQM that addresses these issues should identify both frequency of violations of validation rules by data validation, and frequency of out-of-range value reporting by field. If significant numbers of grantees commit the same reporting violation, the DQM should note any specific pattern of reporting that contributes to this violation; similarly, in the case of frequently occurring out-of-range value reporting, the DQM should note the patterns of out-of-range values that are reported. Also, when violations occur, the memorandum should identify grant projects that commit these violations. In order to support this work, the Program Office will supply the Contractor with a document enumerating all data validation rules (which range between 30 and 60 per program), and a printable version of the data collection instrument, which provides ranges (or sets) of valid values by field.

The DQM shall also include a section in which the Contractor highlights findings and provides recommendations for improving the validity, reliability, and usefulness of TRIO’s APR data. Such recommendations shall address, as needed, the potential addition of new data validation rules.

In addition to the requirements above, for each program’s DQM, the Contractor shall:

· summarize the number of grantees, participants, and major subgroups of participants (e.g., current and prior participants);

· identify, as appropriate, grantees or groups of grantees whose data or performance appears problematic in some important respect (e.g. grantees that served fewer than 90 percent of the participants that they were funded to serve);

· provide observations, if necessary, on fields essential for calculating prior experience points; GPRA and other performance measures; and certain priorities of the competitions; and

· respond to specific questions that TRIO staff members may raise about the data.

Subtask 3.3: Data Verification Study (Optional)

The program specific Data Quality Memoranda (DQMs) produced in Task 3.2, addresses data reliability issues at both the program and grant project levels, and other issues associated with data quality. However, the memoranda are not designed to address data validity concerns: i.e., independent verification that data submitted in annual performance reports is accurate and consistent with existing external documentation. The Department may opt to require a study that will estimate the extent to which student level data validity concerns are prevalent within the APR collection.

The Department will select a single TRIO program that the Contractor shall investigate upon receipt of the student record file following an APR collection; the investigation shall be designed to estimate the prevalence data validity concerns during the prior data collection. The study shall take advantage of the fact that grant programs are required to maintain documentation for the records that they submit to the Department.

The Contractor shall verify student record level information submitted in the prior year APR by comparing it to independent documentation maintain by the grant project. The Contractor shall work with the Department to determine a sampling strategy (including a strategy for sampling projects, students within projects, and fields within the student record file to verify), and a data verification strategy, in order to make feasible such a study. The Contractor shall submit a study design memorandum that summarizes this information. The memorandum shall make explicit the fields that will be evaluated, and the number of projects that will be sampled, and the number of students within each project that will be evaluated. The study design memorandum shall also provide a list of tables that will be provided to the Department as part of a final report (see Subtask 3.4). The list shall provide detail about the structure of each table, including the columns of information that will provided.

Note that the Department will not support travel as part of this subtask; therefore, the proposed data verification strategy should assume remote communications with the grant projects in the study.

At the end of the contract period, the Contractor shall submit a study status memorandum to the Department that enumerates the sampled grant projects for which the Contractor has completed the verification process, which sampled grant projects the verification process remains ‘in progress’, and which sampled grant projects the verification process is not yet initiated. The study status memorandum shall address general issues related to the quality of information collected for each grant project which has completed the verification process. Such issues include, the types of documentation supplied, by sampled field, and the frequency of missing supporting documentation by sampled field.

Subtask 3.4: Final Data Verification Final Report (Optional)

The Contractor shall present findings from the data verification study (launched in the prior year).

The report shall describe the sampling strategy, the data verification methodology, and identify sampled projects and students. Because the report will contain student level information, care shall be taken that the memorandum handled, stored, and transferred according to Department and Federal Standards for documents with potentially sensitive personally protected information.

The report shall provide fully populated information in the tables describe in Deliverable 3.3.1 (from Subtask 3.3). The report shall summarize findings from the study and provide a commentary on the types of general inferences that are warranted regarding the prevalence of data validity issues for the grant program.

Deliverables:

3.1.1 Memorandum on Global Measures of Reporting Reliability

3.2.1 TS Data quality memorandum

3.2.2 EOC Data quality memorandum

3.2.3 SSS Data quality memorandum

3.2.4 UB/UBMS Data quality memorandum

3.2.5 VUB Data quality memorandum

3.2.6 McNair Data quality memorandum

3.3.1 Study design memorandum (optional)

Task 4: Management of Longitudinal Data Files

Background and purpose for longitudinal data files. The Contractor for the TRIO Analysis contract shall manage the longitudinal data files for each of the four APR data collections that require the submission of student record files (i.e., the collections for (1) Student Support Services, (2) Upward Bound/Upward Bound Math-Science, (3) McNair, and (4) Veterans Upward Bound). These four longitudinal data files each contain records for students that grant projects have tracked and reported-on over multiple years in discrete annual updates. The four files vary, in terms of the year in which the file was established, and the size of the file. When the data is arranged in a flat (or “wide”) format, each record represents a unique student (within a grant project) and each variable represents a field of information collected in a given year. Table 2 presents, by TRIO program, the years of data collection and the number of records and number of variables in the longitudinal data files as of (date).

Table 2: Characteristics of TRIO student-level longitudinal data files, by program (fall 2019)

2018-19

Student Support Services 1998-99 to 2018-19

1,964,316

1,061

Upward Bound/Upward Bound Math-Science 2000-01 to 2018-19

536,397

1,904

A unique student case number identifies each student record within each of the four longitudinal data files. Note that if one student is served by multiple grant projects, that student will have as many records as they have grant projects that served them. Therefore, a student case number identifies a student within a project, but one individual may carry multiple case numbers.

The longitudinal data files are used for the following purposes: (1) Tracking the performance of both the grant projects (through, for example, prior experience points), and grant programs as part of the Government Performance and Results Act (see Task 5); (2) Research (see Task 6); (3) evaluation of data quality of the most recent APR data collection (see Task 3); and (4) promoting accuracy of future APR submissions (see Subtask 4.3).

Subtask 4.1: Annual Updates to the Longitudinal Data File

The Contractor shall update each of the four longitudinal data files [(1) Student Support Services, (2) Upward Bound/Upward Bound Math-Science, (3) McNair, and (4) Veterans Upward Bound)] on an annual basis with the student record files that grant projects submit to the Data Collection Contractor during the annual APR cycle. The Contractor might encounter some complications when matching data from the APR student record files to the longitudinal data files. In the event of a complication in matching a record (e.g., a grant project submits two records with identical student identifiers), the Contractor shall add flags for records with matching complications (by type of complication).

Data matches will be based on the unique student case number field. However, several additional fields of information should also remain identical in matched records, including: (1) last name, (2) first name, (3) date of birth, and (where applicable), (4) cohort. The Contractor shall make certain that the values in all these fields align. If cases arise in which the values in these fields do not align between records that have been matched on the student case number in the longitudinal data file and the student record file, then the case should be evaluated, and adjustments should be made to the student record file to bring the record into full alignment (prior to matching). A flag should be added to the case to document these adjustments.

The Contractor shall evaluate record integrity. If serious integrity issues arise with a record, and it becomes clear that the record does not represent a valid, unique individual, the Contractor should flag the record for serious integrity issues. The Contractor should also flag record integrity issues that are less serious (i.e., the record likely represents a valid case, but contains data integrity problems).

New cases in the APR student record file that should not link to existing cases in the longitudinal data file will arrive to the Contractor with case numbers assigned by the data collection Contractor. The TRIO Analysis Contractor shall evaluate newly assigned case numbers to ensure that all case numbers are unique and conform to all rules regarding case number assignments. Any problematic case number assignment for new records needs to be resolved and may require case number reassignment for new records. In the event of a newly assigned case number, the Contractor shall create a flag for this reassignment.

Finally, for the pre-college service programs (UB/UBMS and VUB), the Contractor shall add fields of information regarding postsecondary enrollment and postsecondary degree completion. The Contractor shall gather this information from either the National Student Loan Data System or the National Student Clearinghouse (or both). When a student is identified as having enrolled in (or earned a degree from) a postsecondary institution, the Contractor shall record the Integrated Postsecondary Education Data System (IPEDS) Unique/Unit Identification Number (UNITID) for that institution so that institutional-level data from IPEDS can be linked to the student record. The longitudinal file shall be updated with the institutional sector (public/private non-profit/private for-profit) and whether-or-not the institution awards a four-year degree.

The Contractor shall prepare and deliver a brief memorandum for each of the four data files that documents when updates have been completed. The memorandum shall report the number of cases that were matched to existing records, the number of new cases added, and the number of cases in the annual student record file that presented complications for matching by type of complication. If necessary, the Contractor shall create a typology of matching complications. The memorandum should address the number of new case numbers generated. Finally, the memorandum should summarize how many records have concordant reporting between the APR and external data source regarding post-secondary enrollment and regarding degree completion. Similarly, the memorandum should summarize how many records have discordant reporting between? the APR and the external data source regarding this information.

Subtask 4.2: Updates to Database Backbone of the Electronic Codebook

Each of the four longitudinal data files is supported by an Electronic Codebook (ECB) which serves as a data extraction tool facilitating user access to data from the file. The ECB allows users to extract data in the following four formats: SPSS, SAS, STATA, and tab delimited (Excel compatible). The ECB serves as data dictionary, data filtering tool, field and records selection tool, and file format converter, as well as providing other functionality. Every annual update to the longitudinal data files requires consistent updates to the Microsoft Access database backbone that the ECB draws from. After each annual update to the longitudinal data file, the Contractor shall append the data table(s) within the database. These tables serve not only to establish elements for the Data Dictionaries that the ECB can generate, but also as sources for elements of data extraction code that the ECB can generate.

The Contractor shall submit each of the four updated electronic codebooks to the Department.

Subtask 4.3: APR Match File

The preparation for each annual APR data collection includes the integration of an APR “Match File” into the on-line APR data collection system. The Data Collection Contractor uploads the APR “Match File” onto the platform that houses the APR Data Collection system, and the system provides a link to this file. The APR match file serves several critical functions supporting the data collection for all continuing grant projects (i.e., all grant projects except first-time, first-year grantees). Most importantly the Match File provides the basis for student record file submission requirements for each year’s APRs. For example, each year grant projects must provide record updates for every single individual listed in the APR Match File; grantees must also provide identifier information in their APR student record file that aligns with identifier information in the Match File. If a grant project does not meet these submission requirements, the system will reject their APR submission.

The TRIO Analysis Contractor will support this process by delivering an APR Match File for each of the four collections that include the submission of a student record file. In general, the APR Match File, will strongly resemble the data that the Data Collection Contractor had passed to the TRIO Analysis Contractor after the prior year’s annual data collection. However, the Match File will not include prior participants who have been tracked for a number of years specified by the Department. The Match File will also reflect all data improvements that the TRIO Analysis Contractor has implemented in meeting the requirements of Subtask 4.1 (e.g., removal of invalid cases, reassignment of invalid case numbers (etc.)). The Data Collection Contractor will provide exact specifications for the Match File and may include providing null values in certain fields.

The TRIO Analysis Contractor should anticipate that the level of effort required to provide these match files should be the equivalent to reviewing the file specifications requested from the Data Collection Contractor and executing a data extraction from the electronic codebook developed in Subtask 4.2. Some additional effort might be needed to accommodate continuing grant projects that did not submit an APR in the prior year. In such cases, the Match File should be drawn from the most recent year in which the grant project submitted data. Such cases are rare. The Department will notify the Analysis Contractor about any cases which would require this type of mitigation at least one month prior to when the Match File is due.

Deliverables:

4.1.1 SSS longitudinal data file, and annual update completion memorandum

4.1.2 McNair longitudinal data file, and annual update completion memorandum

4.1.3 UB/UBMS longitudinal data file, and annual update completion memorandum

4.1.4 VUB longitudinal data file, and annual update completion memorandum

4.2.1 Updates to database backbone for SSS Electronic Codebook, and associated documentation

4.2.2 Updates to database backbone for McNair Electronic Codebook, and associated documentation

documentation

Task 5: Government Performance and Results Act Reports

Background: GPRA requires the Department to report on a set of performance and efficiency measures for each of its programs and to report findings to the Office of Management and Budget (OMB) each year; the Department also aspires to provide these findings to the general public on its Web site, with results shown both at the aggregate level for each of the TRIO programs, and by program at the grantee level. Current performance and efficiency measures for each program are found on our Web site at the links provided below.

Please note that the performance measures for the UB and UBMS programs are the same. The Department currently does not have GPRA measures for the VUB program but will be initiating VUB GPRA reports in the Base Period of this contract cycle.

Reports currently posted on ED’s Web site may be found at the following addresses:

https://www2.ed.gov/programs/trioupbound/grantee-level.html (UB and UBMS) https://www2.ed.gov/programs/triostudsupp/performance.html (SSS) https://www2.ed.gov/programs/triomcnair/performance.html (McNair) https://www2.ed.gov/programs/triotalent/grantee-level.html (TS) https://www2.ed.gov/programs/trioeoc/grantee-level.html (EOC)

Future changes in measures, or additions to measures, are possible for some programs.

The performance metrics that are presented in the GPRA reports are rates with a simple numerator and denominator. The reports present these rates at the national level (i.e., the “program level”), and at the grant project level. At the grant project level, some rates are based on very small counts (either in the numerator or in both the numerator and the denominator). In some cases, these small rates represent a challenge relative to data non-disclosure requirements wherein the Department may not disclose or potentially reveal information about individual students when reporting data. Prior to public release, the Department’s Disclosure Review Board must certify in a memorandum that the data is “safe to release”.

Subtask 5.1: Data Non-Disclosure Methodology

The Contractor shall produce a memorandum that describes how the GPRA reports will comply with Federal and Departmental Data Nondisclosure rules. The Contractor shall review Departmental guidance and rules addressing data non-disclosure with particular attention to rules that emanate from the Family Educational Rights and Privacy Act. The Contractor shall also review the 2019-20 GPRA reports produced by the previous Contractor serving the TRIO Analysis contract in the Summer of 2021, and determine the extent of data nondisclosure challenges in production of GPRA reports for public dissemination. The Contractor shall develop methods for managing compliance with data nondisclosure rules. The memorandum shall include a detailed description of the methods, as well as findings from the review of the prior year GPRA reports. Since non-disclosure challenges may vary across the five different 2019-20 GPRA reports, it may be the case that different non-disclosure methodologies may be necessary for the different reports. In this event, the memorandum shall be explicit in how different methodologies should be applied to the different reports.

[Optional Deliverable]. The Contractor shall obtain a “Safe-to-Release” memorandum from the Department’s Disclosure Review Board (DRB), for the 2019-20 GPRA reports produced by the prior incumbent serving the TRIO Analysis contract. In order to secure a Safe-to-Release memorandum, the Contractor would need to: (1) submit to the DRB the proposed methodology for managing compliance with non-disclosure issues, (2) ensure that 2019-20 GPRA reports comply with the proposed methodology, (3) complete DRB review board documentation that is required for a non-disclosure review, and (4) attend (virtually) the DRB meeting when the evaluation of data non-disclosure for 2019-2020 TRIO GPRA Reports are on the agenda.

Subtask 5.2: 2019-20 GPRA Reports for Web Posting (Optional)

Just prior to the Kick-Off of the new TRIO Analysis contract cycle beginning at the end of September 2021, the previous Contractor serving the TRIO Analysis contract will have delivered draft 2019-20 GPRA reports for five TRIO programs. These deliverables will be transferred to the new Contractor as part of Task 2. In order for these draft reports to be ready for public release (i.e., web posting), they must be certified by the Department as being compliant with standards consistent with Section 508 of the Rehabilitation Act of 1973 (as amended), which can be found at 36 C.F.R. § 1194.1 & Apps. A, C & D. These standards require that Federal agencies ensure that electronic information be accessible to persons with disabilities (i.e. “508 compliant”). Additionally, the reports must be certified as ‘safe to release’ by the Disclosure Review Board which evaluates whether the reports contain information disclosure risks (see Subtask 5.1).

If the Department chooses to exercise Subtask 5.2 in the Base Period, then the Contractor shall deliver for each program requiring a GPRA report, a final document that is certified 508 compliant and ready for web publishing.

Subtask 5.3: 2020-21 Draft GPRA Reports

The Contractor shall submit six program-level reports that provide performance metrics at both the national-level (i.e., for the grant-program as a whole) and reported at the grant project-level. The reports must include detailed notes on calculation sources and methods. As necessary, the Contractor shall discuss the context of the data and shall illuminate any significant changes over time. To provide a general idea of length, the program-level reports produced in the past have been as long as eight pages for programs with participant-level reporting but have been briefer for TS and EOC.

The Contractor shall also prepare reports on performance and efficiency measures suitable for posting on the Department’s Web site for public use. Tables in these reports shall include both program- and grantee-level data (again based on the grantees’ 2020–21 APRs) and shall be accompanied by detailed explanatory and contextual material that is thorough, but suitable for a general audience. The Contractor shall include discussion of the implications and limitations of the data and how the figures were derived. Tables will vary with the number of grantees in the program; explanatory text for programs with participant-level reporting has been up to 12 pages.

Deliverables:

5.2.1 TS Grantee-level GPRA tables and narrative for Web posting, 2019-20 (optional)

5.2.2 EOC Grantee-level GPRA tables and narrative for Web posting, 2019-20 (optional)

5.2.3 UB/UBMS Grantee-level GPRA tables and narrative for Web posting, 2019-20 (optional)

5.2.4 SSS Grantee-level GPRA tables and narrative for Web posting, 2019-20 (optional)

5.2.5 McNair Grantee-level GPRA tables and narrative for Web posting, 2019-20 (optional)

Task 6: Research and Reporting Subtask 6.1: Tabulations and Small-Scale Analyses

The Contractor shall plan to prepare up to ten additional analysis reports on various facets of the TRIO programs or statistical tabulations of program data. As the need arises for these reports or tables, the Contractor shall discuss with Departmental staff the purpose of the reports, the data sources, and the policy issues involved.