Draft_PWS_-_HSLS_2nd_Follow-Up_8-16-2013.docx

Performance Work Statement High School Longitudinal Study of 2009 (HSLS:09) Second Follow-up

1. Introduction

IA. Purpose of Study

The United States Department of Education has a requirement for a large, complex data collection. This data collection, the Second Follow-up of the High School Longitudinal Study of 2009 (HSLS:09), is designed to continue the series of longitudinal data collection efforts that began with the National Longitudinal Study of 1972 (NLS-72) and was followed by High School and Beyond (HS&B), the National Education Longitudinal Study of 1988 (NELS:88), and the Education Longitudinal Study of 2002 (ELS:2002). HSLS:09 began with a ninth grade cohort in 2009 and was followed by the First Follow-up in 2012. A shorter update collection was conducted in 2013. The subject of this performance work statement, the Second Follow-up student survey, will be conducted in the spring of 2016. The Second Follow-up of HSLS:09 will collect data elements concerning family background, secondary school experiences, post-high school experiences including postsecondary education activities, work force activities, and the experiences of high school dropouts. The method of data collection will be internet, telephone, and personal interviews. This study will culminate in a First Look report and documented data files.

The work will be monitored by the Institute of Education Science’s (IES) National Center for Education Statistics (NCES) within the U.S. Department of Education (ED).

IA.I. Legislation

The Longitudinal Studies program at NCES is authorized by law under the General Education Provisions Act (GEPA) section 406(b), as amended, (20 USC, 122le-1):

'The purpose of the Center shall be to collect, analyze, and disseminate statistics and other data related to education in the United States and in other nations. The Center shall: collect, collate, and from time to time report full and complete statistics on the condition of education in the United States; conduct and publish reports on specialized analyses of the meaning and significance of such statistics;

IA.2. Program Overview

HSLS:09 is the fifth secondary school-based longitudinal study. The earlier studies include: ELS:2002, followed through 2012; NELS:88, followed through 2000; HS&B, followed through 1986 (senior cohort) and 1992 (sophomore cohort); and NLS:72, followed through 1986. Further information concerning HSLS:09, ELS:2002, NELS:88, HS&B, and NLS-72 are available on NCES’s web page, http://nces.ed.gov/surveys/slsp.

I.B. Quality Control

NCES is firmly committed to determining how well each project meets 1) the project goals, 2) the quality of data collected, and 3) implications for future projects. Project quality control should be integrated into every Phase of the project. For instance, potential problems should be identified early so that possible solutions are tested as part of the field test. Item reliability and validity tests for questionnaires and student cognitive tests must be built into the data collection efforts, non-response analyses should be conducted as part of the final weighting process, and survey methodology and sample design effects should be presented for use by all data users. Thus, as in other NCES projects, statistical quality control shall be incorporated into each stage of this project as appropriate.

I.C. NCES Standards

All work conducted under this contract must at minimum meet the standards and guidelines set forth in the 2002 revised statistical standards and guidelines for the National Center for Education Statistics (NCES) and as amended by standards implemented since that time. The NCES Standards are available on the NCES web site: http://nces.ed.gov/

II. Scope of Work

Independently and not as an agent of the United States Government, the Contractor shall provide all personnel, materials, services, and facilities necessary for the project and perform the tasks as described below.

The period of performance for the Second Follow-up of HSLS:09 is 48 month contract(60 months if optional Tasks 5 and 6 are exercised). This study shall have five major components. The first encompasses all initial management and review tasks, including overall project quality control (Task 1). This component encompasses the entire period of the project. The second shall be panel maintenance beginning in 2014 to locate as many HSLS:09 sample members as possible using Web, CATI, and CAPI methods (Task 2). The third component involves the conduct and reporting of the field test for the second follow-up (Task 3) from data element development through OMB clearance, data collection and file creation, and field test reporting for the second follow-up survey. The fourth component involves the conduct of the full scale survey (Task 4) and includes obtaining cooperation, sample selection, administration of student questionnaires via CATI/CAPI, data file development (including weighting and creation of derived variables), production of an electronic codebook (ECB), a user’s manual, methodology reporting, and a First Look report.

The Scope of Work for the contract also includes two -optional tasks:

Optional Task 5: Financial Aid Records. The contractor shall collect financial aid information for all sample members who attended postsecondary institutions from the postsecondary institutions they attended, and, for those who received federal student aid, the federal student financial aid system. This optional task shall include the preparation and analysis of the student financial aid information collected, which includes: 1) contacting the postsecondary institutions to obtain the required student aid data, 2) the electronic encoding or transcription of this data into an encoded form, 3) the development of data files and user documentation, 4) the creation of composite variables and the imputation of missing data, and 5) the creation of restricted-use data files and user documentation. The methods employed to collect and code the financial aid data will be developed and evaluated in a field test with the HSLS:09 field test sample.

Optional Task 6: Postsecondary Transcripts. The contractor shall collect, code, and analyze, postsecondary transcripts for all HSLS:09 respondents who have attended a postsecondary institution. Transcripts for all postsecondary institutions attended between the end of high school and the interview date of the sample member survey shall be collected, including both the undergraduate and graduate education of the sample members. This optional task shall include: 1) contacting the postsecondary institutions to obtain the transcripts, 2) electronically coding the transcripts received, 3) creating transcript weights to adjust for transcript nonresponse, 4) creating composite variables for public-use, 5) the development of data files and user documentation, and 6) preparation of a First Look report. The methods employed to collect and code the postsecondary transcripts will be developed and evaluated in a field test with the HSLS:09 field test sample.

Description of Tasks

TASK 1. MANAGEMENT

The contractor shall manage the collection of HSLS:09 Second Follow-up data in an efficient manner that fosters communications with staff, the Contracting Officer’s Representative (COR), and potential users of the data.

Subtask 1.1 Post Award Meeting

Within the first week after contract award, the contractors, project director, and other key project staff as identified in the proposal shall hold a meeting with the COR and other appropriate NCES staff to review the overall contract tasks, to identify potential problems and possible solutions, and to discuss areas of concern related to the proposed project staffing plan and other management requirements. The primary purpose of this meeting is to refine the management, staffing, and scheduling plans. These refinements are not to alter the specifications of the contract, but to provide management information for use by both the contractor and the government in monitoring the work to be performed. This meeting is also to help the contractor to make use of the experience and materials that NCES staff has gained over the years.

Subtask 1.2 Schedules

Due to the uncertainties of large data collection projects, the contractor shall develop and maintain a detailed schedule for all activities of the project. The schedule shall be updated monthly.

Deliverables: Schedules

Subtask 1.3 Technical Review Panel

To obtain peer review of project plans and products, and to foster communications with potential users of the data, the contractor shall establish a Technical Review Panel (TRP) of 10 consultants and conduct 2 meetings. The 2-day meetings shall be held in Washington, DC. The contractor shall prepare packets, ship packets to TRP members, prepare and provide meeting materials, and prepare minutes of TRP meetings. The TRP shall work with the contractor to improve plans, products, and user friendliness; however, the TRP does NOT report to or advise NCES.

Subtask 1.4 Monthly Reports

The contractor shall report monthly on the progress made in accomplishing the project tasks, the consumption of funds, problems encountered, and plans for the next month.

Deliverables: Monthly Reports

Subtask 1.5 Integrated Monitoring System

The contractor shall develop and maintain an Integrated Monitoring System (IMS) to foster communications with staff and the NCES COR. The contractor shall design the IMS to routinely produce information about the project and provide access to this information to the NCES COR.

TASK 2. PANEL MAINTENANCE FOR HSLS:09 SECOND FOLLOW-UP

The sample for the Second Follow-up of HSLS:09 shall include all those who have participated in the HSLS:09 Base Year or First Follow-up who have not subsequently permanently withdrawn from the study, or are deceased.

The most recent effort to contact HSLS:09 sample members will be passive updating in the spring of 2014 (through the first follow-up contract), when addresses currently on file were used to contact sample members. Sample members were asked to confirm or update their information, and were incentivized with $10 for completing this address update.

To assist in achieving the target response rate goal for the Second Follow-up, the contractor shall conduct panel maintenance activities to find current members of the HSLS sample. Panel maintenance entails using batch tracing, CATI locating, and CAPI locating to find sample member’s current location and get a positive response from them.

The size of the current HSLS:09 sample is about 23,000.

A database of contact information for each HSLS:09 sample member, including sample member names, addresses, phone numbers, e-mails, SSNs (if provided) as well as contact information for parents/relatives/others associated with each sample member, shall be available from the government at the time of this tracing task. Information in this database will include response status, mode of data collection for respondents, incentive amount provided (if applicable), status and source of the contact information as of the 2013 Update round and 2014 round of panel maintenance.

Subtask 2.1 Panel Maintenance Plan

The contractor shall develop a plan for tracing, locating, and contacting all of the current HSLS:09 sample members. The goal of the plan shall be to reach 95 percent of all HSLS:09 current sample members.

Deliverable: Panel Maintenance Plan.

Subtask 2.2 Panel Maintenance

Though the bulk of the data collection (in survey form and, if the optional tasks are performed, in transcript and financial aid record form) will occur in years 2 and 3 of the contract, panel maintenance of the sample will need to occur at the beginning of the contract term. Starting in 2014, the contractor shall conduct intensive sample tracing using batch tracing (a tracing service), and Web, CATI, and then CAPI methods to locate all those in the HSLS:09 sample not found by tracing.

Deliverable: Panel Maintenance Report

TASK 3. FIELD TEST (FT)

The contractor shall develop all systems for data collection, prepare materials for IMCD/OMB forms clearance, train staff in the methods of data collection, test all data collection systems, and report on the quality of the data collection systems as a field test prior to the full-scale collection of second follow-up data.

Subtask 3.1 FT Sample

The contractor shall survey the first follow-up field test sample (assume 1,000 sample members) for the HSLS:09 Second Follow-up field test sample.

Deliverables: FT Sampling Specifications

Subtask 3.2 FT IMCD/OMB Forms Clearance

The contractor shall prepare the IMCD/OMB Forms Clearance package for submission. The package shall request clearance for both the field test and the full-scale collections.

Subtask 3.3 FT Data Collection

The contractor shall collect the field test data from the FT sample with a target response rate of 90 percent. In addition, the contractor shall determine the amount of response times associated with each data element collected via CATI/CAPI. Finally, the contractor shall re-collect data from a small (50) number of FT sample respondents to determine the quality of the data.

Subtask 3.4 FT Reporting

The contractor shall prepare a report documenting all activities, methods, and outcomes of the field test.

Deliverable: FT Methodology Report

Subtask 3.5 Responsive Design Experiment

The contractor shall design and conduct an experiment, using the field test sample, to investigate ways to improve Responsive Design procedures in NCES sample surveys. The ultimate goal of responsive design is to improve data quality. Areas of investigation may include but are not limited to: incentives (monetary and non-monetary), bias reduction methods, different functions for estimating bias and nonresponse propensity, and sample member tracing and locating. The contractor shall prepare a report documenting the design, procedure, and results of the experiment.

Deliverable: Responsive Design Experiment Report

TASK 4. FULL-SCALE DATA COLLECTION

The contractor shall modify all sampling and data collection systems as needed depending upon the results of the field test. The contractor shall prepare materials for IMCD/OMB forms clearance and shall plan to collect data from the HSLS:09 2013 Update full-scale sample members.

Subtask 4.1 Sample

The contractor shall develop a sample frame for the HSLS:09 Second Follow-up full-scale sample.

Deliverables: Student and Dropout Sampling Specifications

Subtask 4.2 IMCD/OMB Forms Clearance

The contractor shall prepare the IMCD/OMB Forms Clearance package for submission. The package shall request clearance for the full-scale collection.

Subtask 4.3 Data Collection

The contractor shall collect the full-scale sample data with a target response rate of 90 percent.

Subtask 4.4 User Manual with Codebook

The contractor shall prepare a report documenting all activities, methods, and outcomes of the full-scale data collection including frequencies for all variables.

Deliverable: User Manual with Codebook

Subtask 4.5 Analysis Plan

The contractor shall develop specifications for derived variables, table shells, and First Look report structure.

Subtask 4.6 Electronic Codebook (ECB)

The contractor shall create derived variables for the estimation of table values and two ECBs (Public-use and Restricted-use).

The contractor shall prepare the descriptive survey report based on the Second Follow-up data.

Deliverable: First Look report for Second Follow-up

OPTIONAL TASK 5. STUDENT FINANCIAL AID RECORDS COLLECTION

The contractor shall collect and code data from the administrative records of postsecondary institutions and the federal student aid system on the institutional, state, and federal aid that all HSLS:09 sample members who enrolled in any type of postsecondary education (4-year, 2-year, or less-than-2-year, and public, nonprofit, or for-profit) have received during all the years that they have been enrolled in postsecondary education that is not already included in HSLS:09, using procedures developed in the field test for this task.

Subtask 5.1 Student Financial Aid FT Sample

The contractor shall field test data collection procedures for obtaining data on the institutional, state and federal aid received from the institutions attended and the National Student Loan Data System (NSLDS) for the same HSLS:09 field test sample or subsample selected for field testing the transcript data collection during all of their years of postsecondary attendance.

Relevant data from the Free Applications for Federal Student Loan (FAFSA), from the Central Processing System (CPS) shall also be included.

The field test of procedures for collecting student aid data shall begin simultaneously with the transcript field test (Optional Task 6), if funded, after the end of field-test data collection. Once a postsecondary institution is contacted to obtain a student’s transcript, both the transcript and the student’s student aid information shall be obtained, to minimize burden on the institution. All efforts to minimize burden and improve data quality shall be taken during this field test.

Deliverable: Student Financial Aid Field Test Sampling Plan

Subtask 5.2 Student Financial Aid FT

The purposes of the Student Financial Aid Field Test are to develop web-based CADE (computer-aided-data-entry) strategies and procedures for contacting institutions and collecting information about a) the institutional, state, and federal student aid that the HSLS:09 sample members have received during their years of undergraduate postsecondary enrollment from all the institutions that they attended, b) the extent to which such information is available from institutions of different types, and c) how closely the data on the federal financial aid that sample members received while in postsecondary education agree with the data obtained from the NSLDS for all those in the field test sample who attended postsecondary institutions.

The CADE system may include: 1) the direct entry of data by institutional staff into a web-based interface to the CADE system (referred to as self-CADE) ; 2) the submission by institutions of student record information in data files according to specifications (data-CADE); and 3) trained field data collectors who abstract the student record data into the web-based CADE system (field-CADE).

The contractor shall use student aid data and other information gained in the financial aid field test to determine the most efficient strategies for contacting postsecondary institutions to obtain the different types of financial aid data; the extent of variation in grant and loan information that postsecondary institutions have, the formats in which these data are stored and how it could be coded for the purposes of HSLS:09, how flexible the CADE system developed for the purposes of the field test needs to be because of differences in the forms in which postsecondary institutions store historical financial aid data on their students, and the nature and extent of release permissions.

Because a large number of postsecondary institutions will have to be contacted to obtain data for in many cases a small number of former students (two or three), the contractor shall use a receipt control system in the field test, and in the full-scale data collection.

The contractor shall use data and information gained in the field test to compare the accuracy and completeness of data on the federal student aid received by HSLS:09 sample members with data from the NSLDS for the same students from the time they entered postsecondary education to the time they graduated or left postsecondary education.

The field test should also be used to determine the level of detail with which institutional forms of student aid can be collected. These detailed types of institutional aid include scholarships of various kinds, merit- versus need-based grants, tuition and fee waivers, tuition and fee waivers for staff, institutional work-study, and other types, as identified.

In addition, if Optional Task 5 is funded, the contractor shall include brief questions in the field test of the questionnaire to determine the feasibility of collecting data from the sample members on total amounts of employer aid received and private grants or loans, and identifying it separately from aid data obtained from the institutions. In addition, if Optional Task 5 is funded, the contractor shall include brief questions in the field test of the questionnaire for the full-scale study to determine the feasibility of collecting data from the sample members about the financial aid they received from their parents or other persons.

Finally, the contractor shall use the field test results to make recommendations on whether student aid policies of the institution can be collected for inclusion in the final data files.

Based on the data collected in the field test, the contractor shall make recommendations concerning the data on different types of student aid that should be collected from postsecondary institutions in the full-scale student aid data collection versus those that should be collected from the federal student aid system, and how much of data can be obtained with the CADE system. The contractor shall summarize this information in a framework of the types of aid that will be distinguished in the data collection and the sources from which each type will be obtained.

The contractor shall develop detailed procedures in the student financial aid field test for contacting postsecondary institutions to obtain student aid data; and monitoring the quality and consistency of the coded data, including a selective audit process, if warranted. The contractor shall prepare instructional materials for training the staff responsible for monitoring the quality of data received from institutions and the federal government.

The data collected in the pilot test shall be provided in a flat file with a separate file of data dictionary information.

Deliverables: Guidelines for Collection and Coding of Student Aid Data Key/coder and Field Staff Training Materials Student Aid Field Test Data File Student Aid Data Field Test Report

Subtask 5.3 Student Financial Aid Full-scale (FS) Data Collection

Using a CADE system, the contractor shall collect data on the institutional, state, and federal aid received by HSLS:09 sample members who enrolled in postsecondary education for all years of postsecondary attendance from the institutions attended and the federal student aid system. The contractor shall report on the methodologies employed to collect this data, create composite variables, produce source data files for a restricted-use data file and electronic codebook (ECB), produce data file documentation, and produce a First Look report for the HSLS:09 Student Aid Study.

The response rate goal for full-scale student aid data collection shall be confirmation that aid of some form (institutional, state, or federal) was or was not received in 90 percent of the instances where an HSLS:09 sample members attended a postsecondary institution for at least one month.

Deliverable: Monthly reports on progress in obtaining student financial aid data

Subtask 5.4Student Aid Composites

Considering important aid and postsecondary access and persistence issues, the contractor shall develop specifications for 25 or more student aid composite variables based on variables in the student financial aid data file.

Deliverables: Student Aid Composite Specifications Student Aid Composite Imputations

Subtask 5.5 Student Aid Data File Documentation

Data file documentation shall be prepared for users of the student financial aid data that describes the activities, methods, and outcomes of the student financial aid data collection process, including frequencies for all composite variables. The document should describe the methods of quality control employed in coding the data, the coding process, and any information about the types of information about the student financial aid policies of the institution that are included in the data or were used in coding the data received from institutions.

Depending upon the comments of reviewers, multiple revisions of drafts of the Student Aid Data File Documentation may be required.

Deliverable: Student Aid Data File Documentation

Subtask 5.6 Source Data Files for the Restricted-use ECB

The contractor shall create the source data and metadata files that will serve as input for an HSLS:09 Restricted-use ECB applications.

Deliverables: Restricted-use ECB Source Input Files

Subtask 5.7 First Look Financial Aid Report

The contractor shall develop an analytical structure and 10 table shells for a First Look Report on the student financial aid received by the HSLS:09 cohort. This report is needed for the initial data release.

Depending upon the comments of reviewers, multiple revisions of drafts of the First Look Student Financial Aid Report may be required. If Optional Task 6 is also funded, this report may be combined with a First Look Transcript Report.

Deliverables: First Look Report Outline and Table shells First Look Student Aid Report

OPTIONAL TASK 6. POSTSECONDARY TRANSCRIPT COLLECTION

The contractor shall collect postsecondary transcripts for the HSLS:09 sample from the institutions they attended, report on the methodology employed, create analysis weights and composite variables, produce a restricted data file, a source data file for EDAT, data file documentation, and a First Look report for the HSLS:09 Second Follow-up Postsecondary Transcript Study.

The response rate goal for the transcript study shall be the collection of a complete transcript for 90 percent of all transcript instances in the HSLS:09 sample, where a transcript instance is the collection of a transcript containing at least one course in 90 percent of the cases where a sample member was enrolled in an institution for more than one month.

Subtask 6.1 Postsecondary Transcript Field Test Sample

A sample shall be selected for the Postsecondary Transcript Field Test, which may be a subsample from the field test sample if that is all that is necessary to develop the needed data collection procedures. The process for selecting these sample members may need to be two stage, including first a selection of field test sample members and then a selection of the postsecondary institutions they attended in order to have sufficient numbers of sample members per postsecondary institution for the pilot test. The full range of postsecondary sectors (public, non-profit, and for-profit institutions) and levels (four year, two-year, and less-than-two year institutions) shall be included among the postsecondary institutions attended by the subsample members, and also institutions with and without computerized transcripts to be able to develop procedures that are compatible and efficient for both situations.

Deliverable: Postsecondary Transcript Field Test Sample Specification

Subtask 6.2 Postsecondary Transcript Field Test Data Collection

The contractor shall field test procedures for collecting postsecondary transcripts from the postsecondary institutions attended by the entire sample or a subsample of the field test sample. The field test for the postsecondary transcript data collection cannot begin until after field test data collection has concluded.

In the Postsecondary Transcript Field Test, the contractor shall determine the frequency with which transcripts can be provided by these institutions, the extent of variation among them in how courses and credits are recorded, the most efficient strategies for contacting the institutions and obtaining the transcripts, and the extent of variation in required release permissions.

In addition, course catalogs shall be collected from the postsecondary institutions that the sample members attended for use in coding the transcripts collected. In some cases, multiple course catalogs may need to be collected.

The data collected shall be prepared and stored in a prototype of the transcript data files to be provided in the final data collection. These files shall include a course level file, and an institution level file, but others may be needed. For some of those in the selected sample who have attended more than one institution, attempts shall be made to collect a transcript from one other institution attended.

The institutions in the Postsecondary Transcript Field Test sample (and in the full-scale data collection) shall be asked about the extent of online courses that are available to students, and about means of inter-institution enrollment, and how, if at all, such enrollment is shown on student’s transcripts.

The transcripts and course catalogs (and other data) collected in the pilot test shall also be used to assess the adequacy of current course content classification schemes for coding postsecondary transcripts, needs for improving the schemes, the completeness of the transcripts obtained, and response rates.

In addition, institutional level information shall be collected from the institutions about course credit and grading policies. A separate institution level file should be constructed containing these institution level data. This information shall be used in the full-scale data collection to provide standardized values of earned credits and grade point averages for each sample member.

Institutional level information should also be obtained about each institution’s policies in granting credit towards degrees from courses taken at other institutions (or online), how these courses are identified on student’s transcripts, and how these other institutions can be identified to use in the coding process.

Using the transcripts collected in the field test, procedures shall be developed for a) contacting institutions to obtain transcripts and course catalogs, b) keeping track of contacts and results, c) consistently and accurately coding the transcripts collected; contacting institutions for the clarification of ambiguities; and monitoring the quality and consistency of the coded data, including selective audit procedures. Because of the large number of institutions that will necessarily have to be contacted for in many cases a small number of students, the use of receipt control system as one aspect of the IMS, or some other method, shall be considered and tested in the field test.

In addition, training materials based on the results of the field test and the experience gained in coding the postsecondary transcripts collected shall be developed for use in training keyers/coders in the full-scale transcript data collection.

Deliverable: Postsecondary Transcript Field Test Data File Guidelines for Postsecondary Transcript Collection and Coding Keyer/coder and Collection Staff Training Materials and Manuals Transcript Quality Control Plan with an Audit Process

Subtask 6.3 Postsecondary Transcript FT Report

A working report shall be written summarizing conclusions reached in the Postsecondary Transcript Field Test concerning: (1) the extent of variation in access to student’s postsecondary transcripts depending upon the collection strategies employed, (2) the resulting response rates as well as resource requirements; (3) the feasibility and possibilities of obtaining course records electronically; (4) the completeness of student transcripts obtained; and (5) transcript coding issues; as well as (6) protocols for coding transcripts.

Deliverables: Postsecondary Transcript Field Test Report

Subtask 6.4 Postsecondary Transcript FS Data Collection

The contractor shall collect and code postsecondary transcripts for all those in the full-scale sample who participated in the Second Follow-up and attended postsecondary education.

Initially, the keyers/coders and transcript collection staff shall be trained in procedures for collecting transcripts, the course classification structure, and procedures for coding courses accurately, using the training materials prepared.

For each course taken, the student level transcript data file shall include but not be limited to the course classification code, the term when the course was taken, the credits earned, the grade earned in the course, the course title from the transcript, and codes for the institution where the course was taken and the institution that provided the transcript. Also included shall be all degrees or credentials earned that are shown on the students’ transcripts, and any honors received. Both literal degree and credential names and a classification code for these degrees, credentials, and honors shall be recorded.

Particular attention should be paid to identifying remedial courses, as well as any indications of projects or other activities undertaken by the student where credits were earned (or not). Courses attempted but not completed should be recorded. Consistent rules shall be developed for coding lump sum credits found on transcripts and decomposing them in relation to apparently similar credits found on other transcripts obtained for the same student from other parts of the same institution or other institution. If college major and minor(s) are consistently present on transcripts, they should be recorded, too, by date.

Course catalogs shall be obtained from the institutions where transcripts are collected, as an aid in coding the transcripts.

In addition, information shall be collected from the institutions about course credit and grading policies to use in standardizing credits and grade point averages. Information should also be obtained about each institution’s policies in granting credit towards degrees from courses taken at other institutions (or online), how these courses are identified on student’s transcripts, and how these other institutions can be identified. A separate institution level file should be constructed containing these institution level data. This information shall be used to provide standardized values of earned credits and grade point averages for each sample member in the final data delivered.

The final response rate for the transcript study shall be the collection of a complete transcript for 90 percent of all transcript instances in the HSLS:09 sample.

Deliverables: Transcript Quality Control Reports

Subtask 6.5 Postsecondary Transcript Analysis Weights and Composites

The contractor shall develop weights for use in analyzing the transcript data to correct for transcript non-response. Non-response bias analyses and other analyses of the weights shall be conducted and included in the data file documentation. Both Taylor Series and BRR weights shall be included.

The contractor shall develop specifications for at least 50 composite variables for the transcript data files.

Deliverables: Postsecondary Transcript Weights Postsecondary Transcript Composite Specifications

Subtask 6.6 Postsecondary Transcript Data File Documentation

The contractor shall produce a Postsecondary Transcript Data File Documentation manual that describes the activities, methods, and data collection outcomes of the data postsecondary transcript data collection and provides guidance to users of the data. The documentation should describe the transcript collection process, data preparation steps, the transcript coding process, methods of quality control employed in coding process, and the framework of supplementary information on the grading, credit, degree requirement and other policies collected from the institutions. This institution-level information should be collected for purposes of preparing the transcript data but shall also be included in the data file prepared for users.

Depending upon the comments of reviewers, multiple revisions of drafts of the Transcript Data File Documentation may be required.

Deliverable: Transcript Data File Documentation

Subtask 6.7 Postsecondary Transcript Source Data Files for User Applications

The contractor shall create the composite variables (40 variables) and source data files to serve as input for an HSLS:09 EDAT application.

Deliverables: Postsecondary Transcript Source Data Files for Restricted-use ECB Applications

Subtask 6.8 First Look Postsecondary Transcript Report

The contractor shall develop an analytical structure and tables shells for a First Look report based on the ELS:2002/12 transcript data. This report is needed for the initial data release. (See Optional Task 5, above, for a description of how this report may be combined with a Student Financial Aid First Look report if both options are exercised.)

Depending upon the comments of reviewers, multiple revisions of drafts of the First Look Postsecondary Transcript Report may be required.

Deliverables: First Look Postsecondary Transcript Report Outline with Table Shells First Look Transcript Report

III. Deliverable Specifications

The specifications for all deliverables to be generated by the tasks described in section II are presented below. Several components of the Integrated Monitoring System (IMS) are included, but the IMS cannot be considered as the sum of its parts-it is more comprehensive. The IMS shall link the NCES COR with project information, of all types.

Unless otherwise specified, deliverables shall not be physically delivered to the NCES COR. Instead, the COR shall be notified of the deliverable's placement within the IMS. An important function of the IMS from the COR's viewpoint is the IMS capability to retrieve deliverables.

III.1 Schedule (Subtask 1.2)

For all activities within the project, the schedule shall detail the key staff assigned, the projected start and end dates, and completion dates. The schedule shall be updated monthly. For any monthly schedule, all completion dates shall be entered and the future three (3) months activities shall be detailed. There are only 2 static dates within the schedule: [1] full scale data collection must begin by January 1, 2016 [3] accepted descriptive survey report must be delivered by July 1, 2017.

Delivery specifications:

Each month (concurrent with the monthly report) an update of [1] off-the-shelf software scheduling file (not the software itself, but rather the activity/date file) and [2] ASCII text file print-out of the schedule shall be included within the IMS.

III.2 List of potential TRP panelists (Subtask 1.3)

Twenty (20) potential panelists shall be listed. Names, addresses, telephone numbers, affiliations, prior TRP memberships, and annotated resumes shall be included. To maintain continuity with the HSLS:09 First Follow-up and 2013 Update at least four (4) of the potential panelists shall be from the HSLS:09 First Follow-up and 2013 Update TRP.

Delivery specifications:

Six (6) weeks prior to the first TRP meeting the list of potential TRP panelists shall be delivered as a Word file. Selected TRP members’ information shall be included within the IMS with off-the-shelf database software for the generation of personalized letters and mailing labels.

III.3 Pre-meeting (TRP) Information packets (Subtask 1.3)

These deliverables are materials (such as data element lists, collection problem reports, etc.) that shall be delivered to TRP members so that they can be prepared for discussion at the meetings. Two (2) TRP meetings of 2-day durations shall be held in Washington, DC (June 2014 and June 2015. Pre-meeting information packets will be delivered to TRP members one (1) week prior to TRP meetings.

Delivery specifications:

Two (2) weeks prior to each TRP meeting, Pre-meeting Information Packets shall be delivered as Word files.

III.4 (TRP) Meeting agendas (Subtask 1.3)

These deliverables provide structure for TRP meetings in terms of times, topics, and discussion leaders. The expected topics for the two (2) TRP meetings are as follows:

1. FT data instruments and sample

2. Second Follow-up full-scale data instruments and sample design

III.5 (TRP) Meeting materials (Subtask 1.3) These materials include documents and draft that are discussed at the TRP meetings.

Delivery specifications:

Two (2) days prior to each TRP meeting, the Meeting Materials shall be delivered as Word files.

III.6 (TRP) Meeting minutes (Subtask 1.3)

These minutes document the TRP discussions.

Delivery specifications:

Three (3) weeks after each TRP meeting, the Meeting Minutes shall be delivered as Word files.

III.7 Monthly Reports (Subtask 1.4)

The monthly report shall detail the progress made in accomplishing the project tasks, problems encountered, and plans for the next month. By attachment, the financial status of the project shall be described, including detailed monthly charges by named staff.

Delivery specifications:

Monthly (on or before the day corresponding to the contract award date) progress and financial reports shall be delivered as Word files.

III.8 (IMS) Document Archive (Subtask 1.5)

All documents (including dated drafts, E-mail communications, meeting notes, etc.) generated during the life of the project shall be stored in the document archives the archive shall identify file name, dates, author(s), format, and key words corresponding to each document.

Delivery specifications:

Monthly updates (corresponding to the monthly reports) using off-the shelf software shall be delivered. A listing of each documents vitals (i.e., file name, dates, author(s), format, and key words) shall be an ASCII text file.

III.9 HSLS:09 Second Follow-up Data Library (Subtask 1.5)

All data files of HSLS:09 second follow-up data collections shall be stored and documented. These include 2009, 2012, 2013 and 2016 full-scale collections and 2008, 2011, 2012, and 2015 field test collections.

III.10 Relational Instrumentation Interface (Subtask 1.5)

The instrumentation for the collection of HSLS:09 second follow-up data shall consist of a PC-based, scripted, windowed computer assisted telephone interview (CATI) and computer assisted personal interview (CAPI) procedures with recursive fills for components surveyed by CATI/CAPI. The CAPI, operating on a laptop PC, shall be undistinguishable from the CATI system in terms of screen text, and skip patterns.

The development of screens for these procedures shall be controlled and documented by a Relational Instrumentation Interface (RII). The main function of the RII is to relationally develop CATI/CAPI screens from the data elements. The essential linkages are as follows:

The CATI/CAPI system shall accept translated text from the RII for [3], [4], [5], [6], and [7]-thus, it shall be scripted. Project staff, not programmer support staff, shall develop the text for the RII that shall be translated into the CATI/CAPI instructions.

Look-up table software for IPEDS codes, industry and occupation codes, major field of study codes, and a key word glossary shall be available within the CATI/CAPI system-thus, it shall be windowed.

Delivery specifications:

Within four (4) weeks of contract award the RII shall be established within the IMS using off-the-shelf database software. The RII shall be updated on a flow basis through FT IMCD/OMB forms clearance and following examination of the results of the field test. The RII database software shall have the capability to produce lists of data elements, skip pattern instructions, preload designations, fill designations, and range checks similar to the listing above as ASCII text files. In addition the RII database software shall have the capability to produce translatable files compatible with the CATI/CAPI systems.

III.11 Production Reports (Subtask 1.5)

The status of all cases in the data collection system, by day of operations, shall be presented in daily production reports. The following status categories shall be displayed:

Web/CATI/CAPI Administrations

Delivery specifications:

One day before the beginning of data collection, the spreadsheet with days as rows and the status categories as columns shall be loaded into the IMS. During data collection operations, the Production Report shall be appended daily.

III.12 Confidentiality Reports (Subtask 1.5)

Confidentiality Reports shall be in two (2) parts. Part 1 shall be a list of all project staff with access to restricted ELS:2002 data. Part 2 shall be a completed (signed and notarized) set of NCES Affidavits of Nondisclosure for all project staff listed (in part 1).

Delivery specifications:

Within one (1) week of contract award, Part 1 shall be included within the IMS as a Word file. Each NCES Affidavit of Nondisclosure shall be completed prior to access to restricted ELS:2002 data. The completed NCES Affidavits of Nondisclosure shall be mailed to the NCES COR in monthly batches corresponding to the monthly report. Part I shall be updated on a flow basis.

III.13 Quality Control Reports (Subtask 1.5)

The quality of CATI operations shall be monitored using the statistical quality control procedures of W.A. Shewhart and W.E. Deming. The operator by time population of CATI activities shall be randomly sampled and monitored for the following events:

The quality of CAPI, from a monitoring point of view, is more difficult to assess. The use of recordings, supervisory actions, and other methods shall be evaluated as methods of gauging the quality of CAPI.

Delivery specifications:

One day before the beginning of data collection operations, the spreadsheet with days as rows and the event categories as columns shall be loaded into the IMS. During data collection operations, the Quality Control Report shall be appended daily.

III.4 Panel Maintenance Plan (Subtask 2.1)

In order to achieve the overall target response rate of 90 percent in the HSLS:09 Second Follow-up survey, an active effort to locate all members of the HSLS:09 sample is needed before full-scale data collection begins in 2016.

Delivery specification:

Starting before the Field Test begins in 2015, the contractor shall actively attempt to contact all HSLS:09 sample members, including the field test sample members, to update their contact information and alert them to the data collection in 2016.

The goal shall be to reach 90 percent of the current, HSLS:09, full-scale sample members.

A plan for phasing and conducting this sample update shall be submitted within six weeks of contract award.

III.15 Panel Maintenance Report (Subtask 2.2)

At the completion of the sample updating effort, a report shall be submitted showing the response rates of different sample groups to the effort to contact sample members, and the quality of the address updates provided.

Delivery specification:

One month after the secession of tracing/locating efforts a working report documenting the response rates of different sample groups, including dropouts, to the locating effort shall be completed and made available on the IMS.

III.16 FT Data Elements and Justifications (Subtask 3.2)

For the data elements included within the RII, justifications suitable for inclusion within the IMCD/OMB package shall be developed.

Delivery specifications:

Two (2) weeks prior to the first TRP meeting, a draft of the FT Data Elements and Justifications shall be delivered as a Word file.

One (1) week after the first TRP meeting, the second draft FT Data Elements and Justifications shall be delivered as a Word file for NCES inter-divisional review.

One (1) week after NCES inter-divisional review, the final FT Data Elements and Justifications shall be delivered as a Word file.

III.17 FT IMCD/OMB Forms Clearance Package (Subtask 3.2)

The FT IMCD/OMB Forms Clearance Package shall include introduction, project justifications, purpose and use of data, descriptions of collection methods, descriptions of efforts to identify duplication, descriptions of the suitability of existing data, methods used to minimize burden, frequency of data collection, adherence to guidelines in 5 CFR 1320.6, consultations, confidentiality, sensitive questions, estimates of costs, estimates of response burden, publication plans and schedules, statistical methodology, sampling, data elements and justifications, and sample notifications. This package shall request clearance for field test and full-scale activities.

Delivery specifications:

One (1) week after NCES inter-divisional review of the FT Data Elements and Justifications, the FT IMCD/OMB Forms Clearance Package shall be delivered as a Word file. During the 120 day clearance period, two (2) revisions for IMCD and OMB concerns shall be delivered as Word files.

III.18 FT SF-83 (Subtask 3.2)

Information needed to complete blocks #13 and #17 on Standard Form 83, which accompanies the IMCD/OMB Clearance package, shall be delivered.

Delivery specifications:

One (1) week after NCES inter-divisional review of the FT…