Draft 2026 - 2028 SED SOW.pdf

CUI

Statement of Work Survey of Earned Doctorates (SED) for the 2026 through 2028 Survey

I. Overview

Purpose

The purpose of this statement of work is to specify the activities necessary to provide the National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation (NSF) a data collection, file maintenance, and analysis capability for the Survey of Earned Doctorates (SED).

Specifically, the Contractor shall:

• Conduct the SED for the 2026, 2027, and 2028 survey cycles (plus tasks during the transition year for the 2025 survey cycle);

• Maintain the Doctorate Records File (DRF), a historical database consisting of SED data on individual research doctorate recipients since 1958;

• Prepare the DRF data incorporating appropriate statistical disclosure limitation methods for use in the NCSES’ interactive data system (https://ncsesdata.nsf.gov/ids) and the SED restricted data system (https://ncsesdata.nsf.gov/rdas);

• Provide analytic support to produce publications to release the SED data for each survey cycle, including the SED Digest, detailed data tables (DTs), and other data products;

• Provide data tabulation support based on data needs within and outside of NSF;

• Conduct research on survey design and methodologies including, but not limited to, eligibility criteria for institutions and doctoral degree programs in the target population;

field of study validation; survey instruments and data collection systems (including Web survey instrument paradata) to improve response rate and data quality, editing and missing data methods; data visualization techniques;

• Prepare analytical and methodological presentations for professional meetings and partner with NCSES to present them;

• Prepare and deliver to NCSES data files and documentation from each SED survey cycle.

Background

The SED, annual national survey of each person who earns a research doctorate1 from doctorate granting institutions in the United States university, has been conducted since academic year (AY) 1958. In AY 2022 (July 1, 2021 to June 30, 2022), surveys were completed by 91.6% of the 57,596 research doctorate recipients. The data collected and maintained through this survey constitute a major national resource, offering information on a critical part of the nation’s scientific labor force that is collected directly from the doctorate recipients themselves.

Federal sponsoring agencies that currently collaborate with NSF and provide funding to conduct the SED include the National Institutes of Health (NIH), the U.S. Department of Education (ED), and the

1 Research doctoral programs are oriented toward preparing students to make original research contributions to knowledge in a field and require a dissertation or equivalent project of work. The PhD is the most common research doctorate degree. Excluded from the SED are doctorate degrees that are designed for practitioners (e.g., most DAs, DMAs, DPHs, EdDs, etc.) and professional certification (e.g., MD, DDS., JD, PsyD, DMin.).

https://ncsesdata.nsf.gov/ids https://ncsesdata.nsf.gov/rdas

National Endowment for the Humanities (NEH). An NCSES staff member will serve as Contracting Officer’s Representative (COR) for all the Federal sponsoring agencies.

Data collected from the survey are added to the DRF, which is a database comprised of SED responses from 1958 forward and of skeletal records (name, university, field and year of doctoral degree) obtained from universities about their doctorate recipients who graduated between 1920 and 1957. The Contractor shall maintain the DRF database and provide an annually updated file (along with appropriate documentation) to NCSES for access by researchers and analysts in academic institutions and other organizations who hold an NCSES-approved restricted-use data license.

The SED provides the universe from which a sample is drawn for NCSES’s Survey of Doctorate Recipients (SDR), a biennial survey collecting information (under a separate contract) on the post-graduation employment-related characteristics of doctorate holders in science and engineering. It is necessary that the SED and SDR be closely coordinated (as described in section 3.4).

The SED utilizes two data collection modes: Web survey and computer-assisted telephone interview (CATI). Institutional Contacts – typically, staff in the Graduate School Office – distribute the uniform resource locator (URL) of the SED Web survey to students at the time they apply for graduation. The overall proportion of respondents using the Web survey reached 98.4% in 2022.

Of the total number of research doctorate recipients each year, approximately 80% complete the SED before or soon after their graduation date. The remaining doctorate recipients require locating and follow-up by the Contractor. The Contractor has used both web and CATI data collection modes to conduct nonrespondent follow-up. In 2022, approximately 1.6% of SED respondents completed the survey by CATI. Proposers are encouraged to submit alternative ideas for data collection that would maintain or increase response rates or create efficiencies and cost and/or time savings.

The SED presents the Contractor with several challenges. It is a large-scale, multi-mode survey of highly educated and mobile persons. Relationships with the actual and prospective survey respondents, and with the Graduate School staff who administer the survey, require careful and continual attention to ensure the viability of this survey. All facets of data collection, analysis, and reporting must be tightly coordinated to meet the high response rate goals (response rates have ranged between 90% and 93% over the past 15 years) and standards of accuracy. Also, the SED is a continuous data collection, and the Contractor will be involved in activities of multiple survey cycles at any one time. Finally, as the SED is a census containing personally identifiable and sensitive information, confidentiality and security of the data are of utmost importance.

Work to Be Performed

This section outlines the major tasks included in the SED task order. The tasks are specified in more detail in Section II.

Task 1. Transition, Project Management Plans, and Procedures. This task includes the transition of the SED between the incumbent contractor and the new contractor, and the establishment of management structure and plans, including issues related to confidentiality, data security, internal and external communication protocols, quality assurance and quality control, and other guidelines under which the project will operate. The new contractor shall work with the incumbent contractor over this time to establish an efficient transition that does not negatively affect the SED response rates or the relationships with participating institutions.

During the last two years of the task order, the Contractor shall plan an orderly transition of records, documentation, computer systems, and procedures.

Task 2. Conduct the 2026, 2027, and 2028 SED. The Contractor shall conduct all the activities necessary to collect and process data and produce a final SED data file. At any point in time the Contractor may need to manage the work activities for three or four different survey cycles, which are based on the academic year defined as July 1st of one year through June 30th of the following year (see Table 1).

Task 3. Technical Support Tasks Related to the SED. The Contractor shall perform technical tasks related to the SED, including the production of multiple online reports of SED data in the form of detailed DTs, the production of special tabulations of SED upon request of data users, and coordination with the SDR.

Table 1 shows the relationship between SED survey cycles and the academic year. Shaded cells represent the full SED survey cycles covered under this task order award.

Table 1: SED Survey Cycles and Academic Years Academic Year Survey Cycle Beginning Ending Data Released SED 2025 July 1, 2024 June 30, 2025 October 2026 SED 2026 July 1, 2025 June 30, 2026 October 2027 SED 2027 July 1, 2026 June 30, 2027 October 2028 SED 2028 July 1, 2027 June 30, 2028 October 2029 SED 2029 July 1, 2028 June 30, 2029 October 2030

In addition, this Statement of Work includes the following appendices to provide additional detail and information related to the major tasks included in the SED task order:

Appendix A: SED Major Deliverables and Milestones Appendix B: SED Survey Cycle and Data-related Deliverables Appendix C: Federal Requirements for Work Appendix D: SED Materials

II. Specification of Work

Task 1. Transition, Project Plans, and Procedures

1.1. Kick-Off Meeting

Within two weeks after the award of the task order, the Contractor’s key project personnel shall meet with the NSF Contracting Officer (CO), the Contracting Officer’s Representative (COR), and other NCSES staff to review and discuss SED contract tasks and deliverables. Discussions will include the overall survey management plan and schedule for the survey implementation, including plans for the transition from the incumbent. Discussions may also include task order clarifications, project expectations, reporting and invoicing requirements, data security, and quality control and monitoring requirements. The Contractor shall prepare and submit the draft agenda for the kick-off meeting three working days in advance for review and approval by the COR. This one-day in-person meeting shall be held in Alexandria, VA at NSF.

Within one week following the kick-off meeting the Contractor shall provide the COR with the meeting minutes describing each issue discussed, the conclusions reached, justification for conclusions, and any next steps.

1.2. Transition Plan and Processes

The COR will provide the new contractor with the incumbent contractor’s draft survey transition plan which was developed under the previous task order. After careful review of the incumbent’s draft transition plan, the new contractor shall develop a more detailed transition plan that includes the status of materials provided by the incumbent contractor (see list below) and any needs by the new contractor for information or materials.

The detailed transition plan shall be submitted to the COR within one month from award of the task order, and shall include a detailed plan for the transition, testing, and operation of all IT systems, and an assessment of the risks accompanying the transition process.

Transition activities will take place between the task order award date and February 2027. Survey operations shall be transitioned in phases based on the timing of the survey cycle subtasks (see Table 2 at the start of Task 2). Transition requires concerted coordination with the incumbent contractor to ensure successful transfer of SED materials, data files, and documentation for all past survey operations and data-related items. Within one month of the task order award date, NCSES will host a meeting between the new contractor and the incumbent. The purpose of the meeting will be to review each contractor’s transition plan, provide the new contractor with relevant background and materials, and convey the importance of a smooth and expedient transition for the SED.

Within two months of the task order award date the new contractor shall meet with the incumbent contractor to review SED operations and work processes, with follow-up meetings as required until February 2027.

Materials to be provided by the incumbent contractor to the new contractor at the start of this task order include but are not limited to the following:

• All software and systems documentation used for the SED, including a flow chart demonstrating the linkages between the various elements of the data collection and data processing systems;

• Detailed and complete inventory of past survey documentation (e.g., data collection instruments, editing specifications, coding processes, imputation procedures, etc.) and data files provided by previous SED contractors, including the incumbent contractor;

• Electronic files containing all programs developed under previous contracts;

• Table specifications and programming code (e.g., SAS, Tau Argus, cell suppressions) used to produce the detailed DTs and other reports for the SED;

• Materials related to the evaluations, methodological research, and statistical activities performed by the incumbent contractor;

• SED historical/archival data files and documentation (e.g., codebooks, file layouts, coding manuals, edit specifications, etc.).

All materials shall be safely stored and/or archived to secure data privacy and confidentiality. In addition, a detailed and complete inventory of all materials received, and all materials developed shall be maintained by the Contractor for transition to the next contractor at the end of this task order. The Contractor is encouraged to identify other aspects of the project that should be included in the transition activities at the end of this task order.

1.3. Survey Management Plan and Schedule

Based on the kick-off meeting discussions and consultation with the COR, the Contactor shall review and make appropriate revisions to the proposed Survey Management Plan and Schedule. The Contractor shall submit a revised Survey Management Plan and Schedule for NCSES review and approval within two weeks after the kick-off meeting. The Survey Management Plan and Schedule must contain the following:

Resource Plans

• Staffing information (with an organizational chart, contact information, and resumes) including task assignments, personnel crucial to successful project outcomes, hourly rates for each proposed staff member, and the number of hours proposed per person per task per year (or a percentage of their time);

• Status of materials provided by the incumbent contractor and identification of any additional materials to move to a fully functioning operation.

Communication Plans

• Plan for establishing communication protocols between the Contractor’s staff and the COR and CO;

• Plan to inform institutions participating in the SED about changes resulting from the transition to the new contractor, including changes in operations and procedures for communicating with the new contractor;

• Plan for releasing data and other products developed under this task order, including a description of the disclosure review process to be followed prior to data or product release.

Project Management Plans and Schedule

• Plan for development of a real-time Information Management System (IMS) that provides up-to-date information on all actions on tasks and deliverables. This IMS shall provide the COR access to real-time information that assists NCSES in managing the project, and shall include but is not limited to the following information: when surveys were sent to each institution; contacts to institutions initiated by the Contractor by mode of contact (paper, e- mail or telephone), date of contact, and reason for contact; contacts initiated by the institutions by mode of contact, date of contact, questions raised and response to questions; up-to-date SED response rates by institution;

• Plan and schedule for a budget and financial reporting system with sufficient detail, by subtask, by SED cycle (AY), and by fiscal year, to ensure careful monitoring of the negotiated costs vs. actual costs for all parts of the SED project;

• Plan and schedule for coordinating with institutions regarding the institutional collection of SED data and mutual efforts to reduce rates of nonresponse;

• Schedule of activities under this task order’s Statement of Work using an electronic package such as Microsoft Project for the development of a work breakdown schedule (WBS) which provides detailed schedules for completion of tasks and subtasks, to monitor progress and interrelated activities.

Survey Coordination Plan

• Information that describes how the Contractor will coordinate with the SDR contractor to provide, in a timely manner, the SED information needed for development of the SDR sample.

Based on the kick-off meeting discussions and consultations with the COR, the Contractor shall review and revise the draft Survey Management Plan and Schedule submitted as part of the project proposal.

The Contractor shall submit a revised Survey Management Plan and Schedule for NCSES review and approval one month after task order award.

1.4. Confidentiality Plan and Data Security Procedures

SED data include personally identifiable and sensitive data (name, address, date of birth, last 4 digits of the Social Security number, employer, salary, etc.) that are collected under a pledge of confidentiality to respondents. Consequently, the handling of SED data requires that special protections be applied.

The Contractor shall be responsible for protecting the confidentiality of data about individuals, as required by the Federal Cybersecurity Enhancement Act of 2015, Privacy Act of 1974 and the NSF Act of 1950 as amended. The Contractor shall also be subject to all federal (e.g., Federal Information Security Management Act (FISMA), Federal Information Processing Standards (FIPS), Federal Risk and Authorization Management Program (FedRAMP)), NSF, and NCSES policies on these matters, as specified by the NCSES Chief Statistician and the NCSES Confidentiality Officer, regarding data collection, data storage and access, and data dissemination and analysis. These policies may change during the period of the task order.

The Contractor shall submit for review and approval by the COR, the NCSES Chief Statistician, and the NCSES Confidentiality Officer a detailed plan for protecting the security of SED data at the Contractor site (including all microdata and data files that are used as part of these Data, such as the Doctorate Records File (DRF)), protecting the security of SED data when transferring/delivering confidential data protecting the confidentiality of SED respondents, protecting the privacy of survey respondents among staff, protecting against staff disclosure of personally identifiable information for survey respondents. and documenting any other confidentiality and access issues for the SED. This deliverable will be called the Confidentiality Plan and Data Security Procedures (CPDSP) and shall be submitted to the COR within one month of task order award. The CPDSP should include (but is not limited to):

• The security of the facility that will house the SED data (including all related microdata and data files that are used as part of these data, such as the DRF);

• The physical and environmental protections of the rooms in which the data will be housed;

• Access control of the facility in which the data are housed;

• Standards that address the physical and environmental protections for individuals who will access the data remotely;

• Protections for the network on which the data are housed;

• Authorization and access controls for the network on which the SED data are housed;

• Network password requirements to access the network on which the SED data are housed;

• Data use agreements;

• Data disclosure procedures and review;

• Other items to be addressed in the CPDSP are detailed in subsequent sections.

Before the Contractor receives any of the data files from the incumbent contractor, the CPDSP must be approved by NCSES. If necessary, the CPDSP can be modified at the discretion of and with approval from NCSES.

1.4.1 Physical Protection

Upon approval by the COR, the Contractor shall implement procedures addressed in the CPDSP that protect the privacy of survey respondents and protect against disclosure of personally identifiable information. The Contractor shall strictly control access to individually identifiable information. The Contractor shall ensure that only authorized personnel with a need to know will have access to such identifiable data.

The Contractor shall observe secure database maintenance practices and strict version control procedures. Both of these procedures should be documented in the CPDSP. The Contractor shall also document confidentiality and access issues for the SED and shall implement procedures for data protection and dissemination in accordance with FISMA and FIPS protocols.

1.4.2 Data Use Agreements

The Contractor shall document and implement procedures to ensure that all SED data collected and processed will be protected from unlawful disclosure and that the data will only be used for statistical purposes. The Contractor shall inform all staff working with identifiable data of the importance and requirements for maintaining confidentiality. A duly authorized representative of the Contractor’s organization with legal authority to bind the organization must sign the NCSES Organization Data Use Agreement (see Appendix C.1). All contractor staff working on the SED (including any subcontractors) must sign the NCSES Individual Data Use Agreement each year, acknowledging their responsibilities to protect the data (see Appendix C.2).

In addition, contractor staff using NCSES data are required to take annual NCSES-mandated training on computer security awareness, data confidentiality, and any other training mandated by

NSF/NCSES, and must provide evidence to the COR of completion of the training. This training will be made available via the Internet or other media as approved by the NCSES Chief Statistician and NCSES Confidentiality Officer.

The signed Data Use Agreement forms, certificates of completion for the confidentiality training, and the list of all signees shall be delivered to NCSES within one month of task order award, and annually thereafter. Before the Contractor receives any of the data files from the incumbent contractor, these steps must be completed: (1) the CPDSP must be approved by NCSES; (2) the signed NCSES Data Use Agreements for Organizations and all individuals on the project must be received by the COR; (3) staff must complete all NCSES-mandated confidentiality training.

Before actual data can be provided to the Contractor during the transition, the Contractor shall provide evidence that they have received approval from their Institutional Review Board (IRB) or other legal authorities from their organization to conduct activities related to the use of the SED data. If the Contractor does not have an IRB, they shall certify that they are following NSF’s Common Rule for the Protection of Human Subjects (see Appendix C.3).

1.4.3 Data Disclosure Procedures and Review

The Contractor shall provide to the COR a nondisclosure statement each year, documenting that all required staff have signed Data Use Agreements and attended confidentiality training and identifying all incidents of the disclosure of confidential SED data. As part pf its CPDSP, the Contractor shall have an incident response plan (see OMB M-17-12, pp. 11-13) and further the Contractor shall notify the NCSES Chief Statistician, the NCSES confidentiality Officer, and COR in writing (electronic transmission is acceptable) within 1 hour of discovery of any unauthorized release of personally identifiable data to anyone other than individuals with signed NCSES Data Use Agreement forms. The Contractor shall annually submit to the Contracting Officer (CO) and COR the nondisclosure statement and statement of actual disclosures (should any occur) of data each December 30th during the task order period of performance and within one month after the end of this task order.

Currently, there is no public use file for the SED. Therefore, the Contractor shall prepare data tabulations and special analyses for the federal agencies collaborating on the SED and for external users, as specified in tasks 2.11 and 3.2. Procedures to handle these requests and protect the confidentiality of SED data must be specified in the CPDSP and, upon approval by the COR. the NCSES Chief Statistician, and the Confidentiality Officer, these procedures must be implemented and carefully followed.

At present, “data suppression” disclosure avoidance procedures are applied to custom tabulations of SED data to protect against the disclosure of confidential respondent information. The suppression procedures are applied in two steps. In the first (primary suppression) step, the numerical values of all unsafe data cells are replaced by the suppression symbol “D.” A second step (complementary suppression) is required if the suppressed data cell values can be algebraically derived from other data cell values (e.g., column and/or row totals) in the data table, or from data cell values in associated data tables delivered as part of the custom tabulation. Complementary suppression also involves the replacement of numerical data cell values with the suppression symbol “D.”

Data suppression disclosure avoidance procedures are also applied to SED data reported in the Science and Engineering Indicators Higher Education Thematic Report; Diversity in STEM: Women, Minorities, and Persons with Disabilities. A second disclosure protection strategy, field aggregation, is used throughout the Research Doctorates Awarded to U.S. Citizens and Permanent Residents, by Race/Ethnicity, Sex, and Fine Field of Study data tables (also known as the REG Tables).

The CPDSP shall include procedures and criteria for determining whether data requests received by the Contractor involve issues of confidentiality or deductive disclosure of personal identity. These procedures shall be implemented upon approval of the CPDSP by the COR, the NCSES Chief Statistician, and the NCSES Confidentiality Officer. However, if the Contractor has any concerns about the possible disclosure risk of a particular data request, that request shall be forwarded to the NCSES Chief Statistician and the Program Director for the NCSES Data Access, Confidentiality, and Quality Assessment (DACQA) area for disclosure review, with a copy to the COR. The Contractor must inform data requestors of the data limitations and need for data suppression to prevent any possible deductive disclosures. The Contractor shall also inform data requestors that there will be an NCSES disclosure review and approval before the requested data can be released. Routine data requests may be handled by the Contractor if these cases are clearly defined in the CPDSP and are agreed to by the COR, the NCSES Chief Statistician, and NCSES Confidentiality Officer.

The Contractor shall also apply data suppression procedures to the DTs and other data products to be produced under this task order (see tasks 2.12, 3.1, and 3.2). Owing to the size and complex design of some tables, and to the interconnectedness of the data reported in separate tables of the DTs or data products, applying suppression procedures manually will be inefficient and ineffective. Instead, the Contractor shall employ Tau Argus or equivalent disclosure protection software approved by the COR, with input from the NCSES Chief Statistician and NCSES Confidentiality Officer, to apply data suppression procedures to the SED data in DTs and other data products. The Contractor shall purchase a software license for the approved disclosure protection software and shall take all training necessary to operate the software effectively. NCSES may request that the Contractor revise the data suppression procedures at any time during the task order.

1.4.4 Protocol for SED Data Release

No release of data in any form for a new survey cycle is permitted until the COR issues an official release authorization to the Contractor. The Contractor shall specify in the CPDSP and implement internal written procedures and policies consistent with the CPDSP to ensure that data collected during a survey cycle are not released before the official NSF or NCSES release date; violation of this task order requirement is grounds for termination of the task order. Data release procedures shall be described in the Survey Management Plan and must be approved by the COR.

Even after data for a survey cycle are approved for release by the COR, there are still limitations on the access to and use of the data and files. The Contractor shall follow the CPDSP as approved by the COR, NCSES Chief Statistician, and NCSES Confidentiality Officer. The SED data are covered under the Federal Cybersecurity Enhancement Act of 2015, NSF Act of 1950, as amended, and the Privacy Act of 1974. As the manager for this system of records under the Privacy Act, the Director of NCSES is responsible for controlling access to the data.

1.4.5 Linking with NCSES Data

With approval from NCSES, the Contractor may link NCSES data with contractor-owned data or third-party data (upon permission from the third-party data owner) as a potential research tool (“Linked Dataset”). All Linked Datasets are protected by a combination of the Privacy Act, NSF Act, and CIPSEA (depending on the specific NCSES Data included in the match). As a result, all confidentiality and security provisions that apply to the NCSES Data also apply to the Linked Dataset.

The Contractor shall follow the procedures outlined in the CPDSP when protecting Linked Dataset.

All such linking shall be for statistical research purposes only, and nothing in this task order shall be construed to permit linking for any nonstatistical or other purpose not authorized herein.

1.4.6 Disposition of Survey Data

A data disposition plan shall be submitted as part of the CPDSP for approval by NCSES. The data disposition plan shall include instructions related to the closeout tasks near the end of the task order (e.g., delivery of certificate of destruction to NCSES upon completion).

1.5 Standards, Guidelines, and Protocols

NCSES standards, guidelines, and protocols related to computer processing, survey standards, data documentation, release of data, quality control, media contacts and NSF publications shall be followed in the work conducted under this task order. All copies of material developed during the conduct of the surveys, including computer programming codes, documentation, computer files, Web files and documentation, etc., are to be transmitted to NCSES, a future contractor, or disposed of in accordance with instructions from NCSES upon completion of work.

All confidential databases and materials with personally identifiable information must be transferred at the completion of the task order to NCSES or the new contractor.

1.5.1. Computer Processing/Security

The success of this project depends upon integrated data collection and case management systems. All tasks requiring computer processing work shall be conducted at the Contractor’s secure computer facility. All computer programs such as those used in various stages of data processing and generating tabulations shall include standard documentation as a general practice. NCSES prefers the use of SAS.

The Contractor shall request approval from NCSES for the use of statistical computing packages other than SAS. All computer software must be portable, that is, non-proprietary software capable of being transported to another contractor. The Contractor shall implement strict version control procedures, back up the data on a regular basis, and have a process for fast, secure recovery of archived data such as verbatim responses (the written responses to questions such as fields of study, other-specify, employer name/address, title of dissertation, etc.).

The Contractor shall comply with all federal requirements relating to information security and privacy, including the NSF Information Security Handbook, Federal Information and Security Management Act, the E-Government Act of 2002, all relevant National Institute of Standards and Technology (NIST) Special Publication (SP) 800 series publications, such as NIST SP 800-53 Rev. 4 (Security and Privacy Controls for Federal Information Systems and Organizations); Office of Management and Budget (OMB) memoranda related to security and privacy; and FIPS publications such as FIPS 199 and FIPS 200 (Minimum Security Requirements for Federal Information and Information Systems).

(See Appendices C.12, C.14, C.15, and C.16.)

1.5.2. Survey Standards and Guidelines

The conduct, processing, and reporting of the SED are expected to be performed according to survey and statistical standards used by Federal statistical agencies. At a minimum, all work performed under this task order shall meet the standards set forth in the documents used as policy for NCSES, NSF, and the OMB (see Appendices C.8 and C.13).

Survey practices shall reflect state of the art survey research methodology, leading to improvements in data quality, timeliness, and cost efficiency. Contractor staff shall be familiar with any relevant Statistical Policy Working Papers2 and the Statistical Programs and Standards from the Statistical Policy Office of OMB.3 The calculation of response rates will follow the Standard Definitions provided by the American Association for Public Opinion Research (AAPOR)4.

1.5.3. Quality Assurance and Quality Control Plans

The Quality Plans for the SED shall demonstrate the Contractor’s commitment to perform quality work. While quality assurance (QA) and quality control (QC) efforts relate to many tasks within the SED, the planning and execution of QA and QC activity throughout the project is of such overarching importance to be considered a separate activity. However, the Contractor shall budget, plan and conduct all QA and QC activities under the appropriate tasks and subtasks.

The objective of Quality Assurance is the early identification and resolution of problems in cost, timeliness and quality of the SED processes, materials, products, and data file. The Contractor shall design and implement QA procedures that are integrated throughout the entire range of SED activities, including all phases of data collection, processing and analysis. The QA procedures shall identify specific, measurable, realistic objectives and functional requirements for the project, including clear assignment of accountability. The QA plan shall include the metrics to evaluate the QA results of all survey activities.

The QA results should be used to provide timely feedback on the affected areas of survey operations to improve the quality of work. The IMS (described in task 1.3, Survey Management Plans and Schedule) shall be structured to help identify potential problems and sources of error, and to be proactive in responding to problems and errors. The QA Plan shall also include risk assessments of any trade-offs involving data quality, the schedule, and the budget.

The QA Plan for the SED processes, files, and databases shall contain general procedures for quality assurance and will evolve as the task order progresses. The specifications shall be reviewed during each survey cycle and revised as needed and in response to NCSES requests. Results of the QA Plan shall be reported in the Survey Methodology Report for each survey cycle.

A QC Plan is also part of this task. The QC Plan shall specify the procedures the Contractor will use to ensure data accuracy for all final products and deliverables and shall designate the roles accountable for the accuracy of a product (including author, reviewer, QC checker, etc.).

The QA and QC plans shall be submitted to the COR within one month of the award of the task order for review and approval by the COR, with input by NCSES Chief Statistician. After approval of the quality plans, the Contractor’s methods and procedures for quality (and general requirements for checking of work) shall be provided to each staff person on the project. Training in specific areas of quality as relevant to each task shall be provided to staff working on that task.

1.5.4. Protocol for Media Contacts

All requests from the news media received by the Contractor concerning the SED shall be directed to the COR. The COR will then coordinate these requests through the NCSES Communications Team and the Office of Legislative and Public Affairs (OLPA) at NSF. The news media include, but are not limited to, the news reporting operations of publications, wire services, general and special interest publications (including scientific journals and association newsletters), radio, television, cable stations, and networks.

The NSF/OLPA/Media Section requires that it be informed of all news and media (including social

2 https://www.fcsm.gov/resources/statistical-policy-working-papers/ 3 https://www.whitehouse.gov/omb/information-regulatory-affairs/statistical-programs-standards/ 4 https://aapor.org/standards-and-ethics/standard-definitions/ https://www.fcsm.gov/resources/statistical-policy-working-papers/ https://www.whitehouse.gov/omb/information-regulatory-affairs/statistical-programs-standards/ https://aapor.org/standards-and-ethics/standard-definitions/ media such as Twitter, Facebook, etc.) interactions and that, when appropriate, the responses be coordinated with them in advance. Contractor contact with the media shall be initiated only after the COR consults with NCSES Communication Team and OLPA. Requests for routine programmatic or scientific information may be handled by the COR, but the NCSES Communication Team and OLPA public affairs officer shall be informed of the inquiry. Requests for non-routine programmatic or policy information on sensitive or potentially sensitive issues shall always be handled by OLPA (see NCSES’ Policy on Interaction with News Media and NCSES Media Guidelines, Appendices C.9 and C10).

1.5.5. Publications Standards and NCSES Review

All tables and reports for NCSES publication or electronic dissemination on the NCSES website shall adhere to NCSES publication standards. The publication standards are available in Appendix C.4.

These guidelines may be updated by NCSES during the course of the task order.

Files delivered for posting on the NSF Website must be compliant with accessibility provisions stated in Section 508 of the Rehabilitation Act of 1973. Guidance on required compliance can be found in NSF’s Web Development Policy and Standards Manual (see Appendix C.17), WCAG 2.0 Level AA and Synchronized Media. The Contractor will be notified immediately of any changes to these requirements.

All tabulations shall be edited for consistency within and across tables. The programming codes used to produce these tables shall be documented and preserved for future reference.

All task order materials shall be submitted in formats and software specified by the COR. To meet publication goals, the COR will typically require at least two weeks for the initial review of a draft report. Several iterations of the product may be necessary before it is submitted into the NCSES review process. Once in the review process, the Contractor shall be responsible for making any required changes from the review. The NCSES publication review process may take 4-6 months for a data release InfoBrief and a lengthier period for detailed DTs, analytical InfoBriefs, or special reports, depending upon the extent of revisions required. Appendix C.4 contains the File Format Guidelines for NCSES Publications. Note that there is one exception to this requirement: materials prepared for special tabulation requests from the SED Federal Sponsoring agencies for their special tabulation requests shall be prepared in the formats specified by each agency at the time of the request.

1.6. Survey Operations Plan (SOP)

The Contractor shall update the Survey Operations Plan (SOP) based on the SOP provided by the incumbent contractor. The SOP shall include a detailed description of operational plans for conducting the full SED survey cycle, beginning with the 2026 SED.

The SOP shall include, at a minimum:

• Procedures for routine checking of eligible doctorate-granting institutions and degree programs;

• Data collection procedures, including which data collection modes are used (and when they are used) and survey follow-up procedures;

• Procedures and materials for communicating with the Institutional Contacts and doctoral students;

• Procedures to assess and reduce survey mode differences, duplication, and respondent reporting errors;

• Procedures and materials for locating nonrespondents, including different techniques that are based on the difficulty of locating the persons;

• Data receipt control procedures for each survey mode, receipt control tracking and reporting, standard tracking reports, response rate calculations;

• Procedures to maximize response rates;

• Data editing procedures (detailed specifications may be appended to the initial approved SOP when available);

• Imputation plans (specific procedures may be appended to the initial approved SOP if necessary);

• Detailed time schedule with deliverable/major milestone dates.

• Contractor staffing and cost estimates

The Contractor shall reiterate in the SOP the detailed quality control procedures from the QC plan for each of the survey operations/sub-tasks noted above. The SOP shall be submitted for NCSES review and approval within three months of task order award. Upon review and approval of the SOP by the COR, the Contractor shall then conduct the survey closely following the SOP.

1.7. Transition of SED Data, Materials, and Procedures to the Next Contractor

The transition at the end of the task order shall be conducted in a timely and cooperative fashion. The Contractor shall be responsible for transferring all materials developed under this task order to the new contractor in a secure manner with a reliable tracking system. All materials shall be delivered in usable form (e.g., without redactions). During the transition (similar to the period at the beginning of the task order), a number of meetings involving the COR, and the old and new contractors may be necessary.

These meetings may include on-site visits to the Contractor’s facility to view current survey operations.

The Contractor shall prepare a Transition Out Plan to ensure an orderly transition to the next contractor (if applicable) at the end of this task order; it shall be submitted for review and approval to the COR one year before the end of the task order. Elements to be included in the Transition Out Plan are, at minimum:

• Documentation of steps in survey development and implementation, to include data collection, data processing, data dissemination, communication protocols with respondents and doctorate institutions;

• Itemization of all systems documentation, historical data files, computer programs, and data systems and files (paper and electronic) used for SED activities;

• Data security maintenance procedures;

• Schedule for the transfer of materials and work.

The Transition Out Plan shall also include a letter, drafted by the Contractor, that informs all institutions participating in the SED of the change in contractors and the schedule for the changeover.

After approval by the COR, the Contractor shall mail this letter to the institutions.

The Contractor is encouraged to identify other aspects of the project that shall be included in planning for transition activities. Challenges for the future of the SED shall also be discussed in the Transition

Out Plan. Because timing for the actual transition is uncertain (due to procurement processing times), specific options may be proposed.

The Contractor shall work with the new contractor to implement the Transition Out Plan as approved by the COR.

Task 2. Conduct the Survey of Earned Doctorates (SED)

The data collection process for the SED is continual, complex, and changing. In each calendar year of operation there will be work performed for surveys in three or four separate survey cycles. In addition, every year survey responses of nonrespondents from earlier survey cycles will be received; these must be edited and processed according to the procedures of the appropriate survey cycle and then added to the DRF. The transition of the SED from incumbent contractor to new contractor occurs in phases.

The Contractor shall begin work on task 2 subtasks as indicated (X) in Table 2 below.

Table 2. SED Cycle Phase-In Schedule by Task 2 Subtask

Task 2 Subtask SED

SED

SED

SED

SED

2.1. Pre-survey activities X X X

2.2. OMB clearance X X

2.3. Prepare survey mailing materials X X X

2.4. Web survey instrument and Interface X X X

2.5. Distribute survey materials X X X

2.6. Universe list and receipt control X X X

2.7. Reference version of paper questionnaires X X X

2.8. Nonresponse follow-up X X X

2.9. File preparation X X X

2.10. Survey methodology report X X X

2.11. Tabulations for Federal sponsors X X X X

2.12. SED data release products X X X

2.13. Project management X X X

2.14. Quality assurance and control system X X X

2.1. Pre-survey Activities

2.1.1. Determine SED Universe of Institutions and Program Eligibility

At the start of this task order, the current list of accredited universities in the United States and its territories or commonwealths that offer research doctoral degrees will be provided to the new contractor by the incumbent contractor, along with the name and contact information for the Graduate Dean and Institutional Contact for each institution. The Contractor shall maintain and update the list of institutions offering research doctorate-granting programs as required under this task. The Contractor must establish procedures for routine checking of eligible degree programs.

An annual comparison of the list of doctorate-granting institutions from the SED and from the Completions Survey of the National Center for Education Statistics (NCES), as reported in the Integrated Postsecondary Education Data System (IPEDS), shall be performed as a check on the SED universe. Web research, the Higher Education Directory, exploratory discussions with the Graduate

Deans of these institutions, the Chronicle of Higher Education, and other sources shall be used to help identify whether any new institutions award what NCSES criteria consider to be research doctorate degrees. Any institution in the IPEDS dataset or identified from other sources but not in the SED shall be reviewed further by the Contractor.

New doctorate-granting programs may also emerge within institutions that are already included in the SED universe, and the Contractor shall also establish procedures to find these programs and ascertain whether they meet the NCSES criteria for research doctorate degrees. At a minimum, an annual comparison of doctorate counts – by institution and broad field of study – from the SED and from the NCES Completions Survey shall be performed by the Contractor. Data for this tabulation will depend upon the preparation of a complete, edited IPEDS file of data from the Completions Survey. Note that the Completions Survey data for doctorates include both research and non-research doctorates (such as some theology, education, and applied psychology doctoral degrees), so this tabulation must be done by similar field and institution to yield information useful for an evaluation of degree and institutional coverage.

In this review, the Contractor shall examine cases where the IPEDS counts for a given institution are higher than the SED counts. In most cases the higher IPEDS counts will be the result of doctorate degrees reported in IPEDS that are already known to be ineligible for the SED (e.g., Doctor of Physical Therapy, Doctor of Musical Arts, etc.). However, in some cases the higher IPEDS count will indicate the presence of one or more new doctoral programs that may be awarding research doctorate degrees that require further investigation by the Contractor. Proposers are encouraged to submit alternative sources or approaches for such benchmarking.

The Contractor shall annually prepare a report (including the tabulation comparing IPEDS Completions data with SED data) that summarizes the universe review of institutions and programs and recommends: (1) new institutions to be included in the SED; (2) new doctoral programs in existing SED institutions to be added to the SED; (3) doctoral programs to be deleted because they are no longer offered by the institution; (4) institutions to be deleted because they no longer confer research doctorates; and (5) institutions that have research doctoral programs but have not yet conferred the first doctorate (which then need to be monitored so as to ensure their first graduates receive SED survey invitation).

This report shall be prepared by March 1 of each year for review by the COR. All new institutions and programs will be approved by the COR before being included in the SED. The Contractor shall prepare the materials to invite the approved institutions/programs to participate in the SED. Both the Contractor and the institution must be sure that the doctoral programs in question confer research doctorates, not practitioner doctorates or first-professional doctorates, before they are added to the SED universe. For the new institutions/programs that will be joining the SED, the Contractor shall establish an SED Institutional Contact (if the institution does not already have one) and shall monitor and assist all new institutions and programs until their successful participation is firmly established.

2.1.2. Review/Update SED Survey Instruments and Materials

The Contractor shall annually review the SED survey instruments to identify survey content and format improvements that could be implemented for the next SED survey cycle. The Contractor shall

(1) evaluate the information on each question’s performance that is gathered during data collection (e.g., from respondent comments, Web survey instrument, paradata, back-coding of “other (specify)” responses, missing or invalid data); and (2) analyze the content of each SED question to determine whether any need additional response options, wording changes, or format changes. The Contractor shall submit a report by February 1 each year, for COR review, which specifies this information, recommends changes to the survey instruments, and describes the implications of those changes.

For each SED survey cycle, the Contractor shall review, revise, and update all SED contact materials for COR review and approval. In the past, these materials have contact letters, reminder postcards, SED information brochures, the Institutional Contact manual…