SOW FOR RFQ.docx

REQUEST FOR QUOTATION NUMBER 1333ND23QNB100582

Statement of Work

Assessment of the NIST Summer Undergraduate Research Fellowship (SURF) Program

BACKGROUND

The National Institute of Standards and Technology (NIST), International and Academic Affairs Office (IAAO), provides advice on international science and technology engagement; serves as liaison with the international science and technology community; manages NIST bilateral and multilateral cooperation; serves as the focal point for foreign visitors and associates, and oversees NIST's cooperation with academia.

The Academic Affairs portion of the International and Academic Affairs Office has the responsibility of being the focal point for dispersing information regarding NIST's academic programs, its cooperative education activities and education outreach efforts. IAAO/AA acts as the liaison for any programmatic questions regarding NIST academic activities.

Among the academic programs for which IAAO/AA retains responsibility is the Summer Undergraduate Research Fellowship (SURF) .

The SURF Program is designed to inspire undergraduate students to pursue careers in science, technology, engineering, and mathematics (STEM), through a unique research experience that supports the NIST mission.

The SURF Program provides opportunities for undergraduate students to engage in world-class research at NIST’s unique research facilities. The Program has captured the interests of the nation’s next generation of scientists and engineers since 1993, starting in the Physics Laboratory. SURF expanded to a NIST-Gaithersburg (MD) -wide effort in 2002, with NIST-Boulder (CO) officially entering into the program in 2009. SURF now encompasses all laboratories and user facilities at NIST and operates at both Gaithersburg, MD and Boulder, CO campuses.

As a nationally competitive program, SURF applications are ranked and reviewed. The program awards approximately 135 fellowships each year. SURF participants contribute to an ongoing research project under the guidance of a NIST scientist or engineer from one of the Institute’s six operating units for 11 weeks. Additionally, a summer-long lecture series exposes SURF students to a sampling of diverse research areas, presented in ways to peak interest.

In 2003, 2004, 2007 and 2009, the American Institute of Physics (AIP) conducted descriptive program assessments utilizing a post-program survey. The assessment instrument asked participants to evaluate various aspects of their experience at SURF and the extent to which the program impacted their educational or career decisions. The AIP provided qualitative reports on how the program impacted decisions of SURF alumna on their continuing undergraduate in Science, Technology, Engineering, and Mathematics (STEM) and anecdotal views on program satisfaction. Additionally, a subset of the SURF alumna participated in a survey utilizing a Vanderbilt Capstone Project, evaluating the Department of Energy’s Student Undergraduate Laboratory Internship program as a comparison group (Foltz, Gannon, Krischmann, 2011).

OBJECTIVE

In order to meet the Federal Government's requirement that all STEM education programs work toward more rigorous program assessments, an objective and metric-based summative assessment of the SURF Program is required.

The Contractor shall provide an independent program evaluation that includes reasonable metrics to measure its effectiveness and to compare these results with the same metrics from similar groups. The evaluation must also document what benefits NIST garners by conducting this program.

SCOPE OF WORK

The Contractor shall evaluate the SURF programs at the NIST Gaithersburg, MD, and NIST Boulder, CO, campuses. The evaluation shall include Programs that occurred between 2010-2014 and 2015-2019. The evaluation must:

1. Collect metrics to assess the benefits of the program to former SURF students (SURF alumna) and to compare these metrics with other Federal entities and other organizations, that host undergraduate research programs;

2. Collect information to assess the benefits of the program to NIST.

Results of the evaluation must, at a minimum, provide NIST with information to document program strengths and successes, recommendations on how to improve the program as well as the participants' experiences, and analyze diversity within the Program.

GENERAL REQUIREMENTS

-Paperwork Reduction Act. As appropriate, structured questionnaires and/or survey documents must be approved consistent with the requirements of this statement of work and the Office of Management and Budget (OMB), Paperwork Reduction Act (PRA), prior to release of any structured questionnaire, survey and/or commencement of any focus group that will involve 10 or more non-federal employees. The Contracting Officer’s Representative (COR) will guide the approval process. The Contractor must participate in the process and provide necessary documentation and/or revisions as required. The PRA approval process is estimated to be 3-6 months, at a minimum.

-Approval of Deliverables. The Contractor shall submit draft and final versions of all required reports to the COR. Upon receipt of a draft report, the COR will provide comments within 14 calendar days from receipt of a given deliverable. The Contractor shall make any needed changes to the report and deliver a final version not later than 15 calendar days from receipt of electronic or written comments from the COR.

-Data Rights. The Government shall maintain unlimited data rights to all raw data and deliverables under this contract, in accordance with the data rights clause incorporated herein.

-Information Technology Security Requirements. Contractor’s handling of Government information must meet the requisite IT security and privacy controls to be compliant with security and privacy requirements defined by applicable federal laws, regulations, policies, and standards (e.g., the Federal Information Security Modernization Act (FISMA), Office of Management and Budget (OMB) Circular A-130, and Federal Information Processing Standards (FIPS) Publication 200), as well as requirements identified in the statement of work.

Collection and storage of NIST data must be done using NIST-managed/administered resources that are authorized to handle controlled unclassified information (CUI), to include data categorized at the high, moderate, and/or low impact levels, in accordance with Department of Commerce/NIST requirements. Reference NIST Special Publication 800-53, Security and Privacy Controls for Information Systems and Organizations. In addition, the Contractor must work with NIST in ensuring that any survey or collection of data meet requirements set forth by the Paperwork Reduction Act and other related Federal requirements.

The Contractor must use NIST-authorized data storage solutions that have been authorized to handle data that is categorized by NIST as high, medium and/or low impact data. In order to meet these requirements, the Contractor must comply with all of the following:

-The Contractor will be provided not more than four (4) NIST-owned computers. Reference the section titled “Government Furnished Equipment;”

-NIST-owned computers are limited to use in the United States only;

-All communication under this contract must be done via a NIST email account;

-The Contractor shall not load any software, applications, and/or databases to NIST-owned computers without the advanced written approval of the Contracting Officer. Any software, applications, and/or databases to be added to NIST-owned computers must be FEDramp approved and/or be assessed, authorized and approved by the NIST/Office of Information Systems Management (OISM);

-All data contained in the final report must be in aggregate form such that no data is attributable to any individual or person.

-NIST Associate Status. Prior to starting specific tasks some Contractor personnel will be required to meet requirements to become a NIST associate. This includes meeting the Department of Commerce (DOC)/NIST’s IT access and use requirements and acquiring and maintaining a Personal Identity Verification (PIV) badge.

Contractor personnel must be U.S. citizens and/or green card holders and must undergo all processes necessary to be vetted as a NIST Associate (NA). Only personnel approved by NIST as NIST Associates are permitted to work on tasks involving data collection, analysis, and/or outreach to survey participants. Individuals approved by NIST as NIST Associates will be identified as such in the contract.

To be approved by NIST as an Associate, individuals must provide personal information for entry in the NIST Associates Information System (NAIS). This information will be sent to the appropriate personnel for processing and to allow the NA preliminary access to the NIST campus and resources. The information collected may also be the basis for further background investigations, as necessary.

The Contractor must work with the Contracting Officer’s Representative (COR), and other authorized NIST personnel to obtain the appropriate authorization for release of NIST data from government PII processing systems (e.g., NAIS, HR System, etc.). All relevant data release authorizations must be obtained prior to receiving access to information about survey participants, or any other information considered by the Department of Commerce as low, medium or high risk.

-Options The statement of work includes a base period line item (Line Item 0001) and one separately priced option line item (Line Item 0001A). Line Item 0001 is the base line item and includes Tasks 1, 2, 3, 4, 6, and 7, to perform an assessment of the NIST SURF Program, with a survey audience of approximately 850 individuals, for calendar years 2010-2014. Line Item 0001A is a separately priced option line item and includes Task 5 relevant to Line Item 0001.

The statement of work includes Option Period I (Line Item 0002). Work required by Line Item 0002 shall be performed during the option period, if exercised, and includes Tasks 1, 4, 6 and 7 for a survey audience of approximately 1,000 individuals for calendar years 2015-2019. Line Item 0002A is a separately priced option line item and includes Task 5 relevant to Line Item 0002.

The Government may exercise the base period separately priced option line item (Line Item 0002A), at its discretion, at the time of award, or at any time prior to the release of the survey instrument during the base period.

The Government may exercise Option Period I, at its discretion, at any time prior to the expiration of the base period. The Government may exercise the separately priced option line item (Line Item 0002A), at its discretion, at the time the option is exercised, or at any time prior to the release of the survey instrument.

SPECIFIC TASKS

Task 1 – Kick Off Meeting The Contractor shall attend a kick-off meeting with the NIST Contracting Officer (CO), NIST’s Contracting Officer’s Representative (COR), as well as other authorized NIST personnel. At a minimum, the Contractor’s Project Manager must be in attendance. Other individuals may be in attendance at the Contractor’s discretion.

The purpose of the meeting is to review all statement of work requirements and identify the information required to be submitted for Contractor personnel to undergo all processes necessary to be vetted as a NIST Associate. Only personnel approved by NIST as NIST Associates are permitted to work on tasks involving data collection, analysis, and/or outreach to survey participants. The Contractor shall not review any survey participant data/information until personnel are confirmed as NIST Associates. The COR shall lead and monitor this process.

The kick-off meeting must commence not later than 15 days after the date of award and be conducted via conference call or webcast. The NIST COR shall schedule the meeting and provide the web link for the meeting.

The Government anticipates that the Contractor will require access to Government furnished data sources prior to development of the evaluation methodology and the survey instrument. Contractor personnel must be fully vetted as NIST Associates prior to obtaining access to any of the equipment or data sources referenced in the section titled Government Furnished Equipment, Information, and Data

Task 2 - Develop the Evaluation Methodology The Contractor shall develop an evaluation methodology which clearly identifies and discusses the purpose of the evaluation and the approach that will be utilized to complete the evaluation and secure the necessary data. At a minimum, the Contractor shall identify the target audience, the size of the audience, intended metrics for the survey instrument, the steps that will be taken to execute the evaluation, response timeline for survey responses, how the survey responses will be analyzed, an outline for the final report, and any other information pertinent to the evaluation methodology.

Task 3 – Develop the Survey Instrument The Contractor shall work with the Government in developing the survey instrument. The survey instrument must yield data in all of the following areas:

-The short term and long-term benefits of the program to former SURF students;

-Comparison of these results against an appropriate comparison group;

-SURF alumni eventually hired by NIST;

-The diversity of the SURF Program as compared to the diversity of the population of undergraduates pursuing STEM degrees;

-Inclusion of under-represented STEM communities with a focus on MSIs.

In developing the survey instrument, the Contractor shall consider all of the points below. These points are not the actual questions to be asked, however, represent the Government’s areas of interest. The Contractor shall develop survey questions based upon these areas of interest.

For the purpose of this evaluation, “short term benefits” are defined as benefits realized by SURF students within 1-4 years after completing the SURF Program. “Long term benefits” are defined as benefits realized by SURF alumna with 5-10 years after completing the SURF Program.

Short-Term Benefits How did the program perform in assisting the SURF students:

-In building relationships with NIST scientists and other students?

-In recognizing the importance of advanced degrees in STEM fields?

-In assisting participants in decisions about their future fields of research interest?

-In increasing their understanding of how scientific research is accomplished?

Long-Term Benefits How did the program contribute to SURF students’ pursuit of advanced STEM degrees and careers?

-What proportion of SURF students obtained undergraduate STEM degrees?

-What proportion of SURF students obtained a doctorate or other advanced degree in STEM or are currently in a doctoral or advanced degree program in a STEM field?

-What proportion of SURF students have been or are employed in a STEM occupation?

Short Term Benefits How did the program contribute to the NIST mission regarding:

-Valuable extra help on research projects?

-Novel and youthful viewpoints on technical challenges?

Long-Term Benefits How did the program contribute to the NIST mission regarding:

-Its value as a recruitment tool for specific technical areas?

-Increasing the number of people in the U.S. pipeline for future STEM jobs?

-Level of satisfaction in sharing knowledge with the next generation?

-Its value in developing NIST ambassadors to share NIST’s work with others?

Diversity Diversity within the STEM community and maximizing opportunities to under-represented STEM communities are among the Government’s highest priorities within the SURF program. The Government requires feedback to determine the following:

-Have numbers of participants from under-represented STEM communities increased or decreased during the assessment period?

-Considering the demographic information of SURF alumni, what degree of diversity is active within the SURF program? Ethnicity, gender, race, and participation of individuals from under represented STEM communities should be considered;

-What actions has NIST taken that have made an impact within under-represented STEM communities?

-What additional actions should NIST take to further the impact within under-represented STEM communities?

-Did the SURF program motivate students to advocate for STEM and/or to engage with other STEM students.

For the purpose of this statement of work “underrepresented STEM communities” refers to races or ethnicities whose representation in STEM employment and STEM education is smaller than their representation in the U.S. population. This includes, at a minimum, Blacks or African Americans, Hispanics or Latinos, American Indians or Alaska Natives, and women.

The Contractor shall draft a survey instrument for review by the COR and other authorized NIST personnel. The COR shall provide comments to the Contractor on the survey. The Contractor must update the survey in accordance with the comments provided.

Upon approval of the COR, the survey instrument shall be forwarded for Paperwork Reduction Act clearance. Reference the General Requirements. The Contractor shall respond and revise the survey instrument as required by OMB.

The Contractor shall not release the survey instrument publicly, or to any party, with the exception of the CO, COR, and other authorized NIST personnel, until notified to do so by the COR. Upon approval by OMB, the COR shall notify the Contractor to release the survey instrument.

Task 4 – Execute the Survey The Government will provide the Contractor electronic spreadsheet records of SURF students for the assessment period.

The Contractor shall create a single electronic record for submission to the COR containing all of the following information for each student: Name; Email; University affiliation(s); Dates that student was in the SURF program.

The Contractor shall create a mass distribution list & work in conjunction with the COR to establish an email response database where all responses will be delivered. NIST will provide the email database and provide access to the Contractor.

The Contractor shall prepare a message to be released with the survey. The message text must be reviewed and approved by the COR prior to release.

The Contractor shall document student names and email addresses that are identified as non-deliverable on the electronic record.

Task 5 – LINE ITEM 0001A SEPARATELY PRICED OPTION LINE ITEM Upon electronic notification that an email is not able to be delivered to the email address on record, the Contractor shall utilize available technology (i.e. phone, email, social media) in an effort to secure a current email address for the student. The Contractor shall document efforts made to contact the student on the electronic record. This action shall be taken for not more than 100 students.

Task 6 - Analyze Responses The Contractor shall monitor the email database for responses. As responses are received, the Contractor shall sort and analyze responses.

The Contractor shall save all responses electronically and analyze responses in accordance with the Contractor’s evaluation methodology. At a minimum, the analysis must capture all of the following based upon survey data obtained:

-The short term and long-term benefits to SURF alumni;

-The short term and long-term benefits to the NIST mission;

-Any weaknesses or issues identified relevant to the SURF Program;

-Areas for improvement relevant to the SURF Program;

-Diversity of the SURF Program.

Task 7 – Comparison to Similar Groups The Contractor shall compare the data received from this survey to data available in the public domain for undergraduate fellowships at other federal agencies. The Contractor shall compare data collected to relevant data available. As available, the Contractor shall document the percentage of students from MSIs and the percentage of students from underrepresented communities as available in data from other federal agencies.

Task 8 – Reporting Requirements

8.1 Progress Reports

The Contractor shall provide a progress report in MS Word format every three (3) months. The report must include all work completed within the reporting period, work anticipated to be completed within the next three month period, any issues identified and steps taken to resolve the issues.

8.2 Final Report

The Contractor shall submit a final report. The report shall be a narrative report, in MS Word format and shall include data to support all conclusions drawn.

The final report shall:

(1) Detail the methodology, including study design, sampling strategy, and data collection methods, including a summary of each task;

(2) Quantify the number of survey participants and respondents;

(3) Discuss the analysis performed and outcomes realized consistent with Task 6;

(4) Provide a summary of the comparison performed in Task 7, including the organizations/agencies utilized for the comparison, specific areas of comparison, and outcomes realized;

(5) Conclusions. Based upon all data collected, discuss how the SURF Program is beneficial to NIST, whether or not it continues inspire undergraduate students to pursue careers in STEM through a unique research experience that supports the NIST mission, and any areas whereas the Program can be improved.

8.3 Presentation

The Contractor shall summarize and present the final report to select NIST management and staff. The Contractor shall prepare all presentation materials and provide the materials to the COR for review and approval not later than five days prior to the scheduled presentation date. Changes must be made in accordance with COR comments. The presentation shall be virtual. The NIST COR shall schedule the meeting and provide the web link for the meeting.

LINE ITEM 0002 - OPTION PERIOD I

SURVEY 1,000 INDIVIDUALS FOR CALENDAR YEARS 2015-2019

Task 1 – Kick Off Meeting The Contractor shall attend a kick-off meeting with the NIST Contracting Officer (CO), NIST’s Contracting Officer’s Representative (COR), as well as other authorized NIST personnel. At a minimum, the Contractor’s Project Manager must be in attendance. Other individuals may be in attendance at the Contractor’s discretion. The purpose of the meeting is to review the requirements for the option line item.

The kick-off meeting must commence not later than 15 days after the date the option is exercised and be conducted via conference call or webcast. The NIST COR shall schedule the meeting and provide the web link for the meeting.

Task 2 – Evaluation Methodology The Government intends that the Contractor utilize the evaluation methodology developed under Line Item 0001, however, the Contractor and the Government must revisit the evaluation methodology, identify any potential changes based upon the initial survey completed and edit the evaluation methodology as appropriate. The Contractor shall provide the revised evaluation methodology to the COR for review and concurrence.

Task 3 –Revisit the Survey Instrument The Government intends that the survey instrument will remain 75-80% unchanged from the survey utilized under Line Item 0001. The Government and the Contractor will review the survey document and identify and discuss any changes based upon the survey results from Line Item 0001. The Contractor shall update the survey to include all required changes.

The COR will confirm with OMB that further review and approval is not required based upon changes made to the survey document. The Contractor shall respond and revise the survey instrument as required by OMB.

The Contractor shall not release the updated survey instrument publicly, or to any party, with the exception of the CO, COR, and other authorized NIST personnel, until notified to do so by the COR. Upon approval by OMB, the COR shall notify the Contractor to release the survey instrument.

Task 4 – Execute the Survey The Government will provide the Contractor electronic spreadsheet records of SURF students for the assessment period.

The Contractor shall create a single electronic record for submission to the COR containing all of the following information for each student: Name; Email; University affiliation(s); Dates that student was in the SURF program.

The Contractor shall create a mass distribution list & work in conjunction with the COR to establish an email response database where all responses will be delivered. NIST will provide the email database and provide access to the Contractor.

The Contractor shall prepare a message to be released with the survey. The message text must be reviewed and approved by the COR prior to release.

The Contractor shall document student names and email addresses that are identified as non-deliverable on the electronic record.

Task 5 – LINE ITEM 0001C SEPARATELY PRICED OPTION LINE ITEM Upon electronic notification that an email is not able to be delivered to the email address on record, the Contractor shall utilize available technology (i.e. phone, email, social media) in an effort to secure a current email address for the student. The Contractor shall document efforts made to contact the student on the electronic record. This action shall be taken for not more than 100 students.

Task 6 - Analyze Responses The Contractor shall monitor the email database for responses. As responses are received, the Contractor shall begin to sort and analyze responses.

The Contractor shall save all responses electronically and analyze responses in accordance with the Contractor’s evaluation methodology.

At a minimum, the analysis must capture all of the following based upon survey data obtained:

-The short term and long term benefits to SURF alumni;

-The short term and long term benefits to the NIST mission;

-Any weaknesses or issues identified relevant to the SURF Program;

-Areas for improvement relevant to the SURF Program;

-Diversity of the SURF Program.

Task 7 – Comparison to Similar Groups The Contractor shall compare the data received from this survey to data available in the public domain for undergraduate fellowships at other federal agencies. The Contractor shall compare data collected to relevant data available. As available, the Contractor shall document the percentage of students from MSIs and the percentage of students from underrepresented communities as available in data from other federal agencies.

Task 8 – Reporting Requirements

8.1 Progress Reports

The Contractor shall provide a progress report in MS Word format every three (3) months. The report must include all work completed within the reporting period, work anticipated to be completed within the next three month period, any issues identified and steps taken to resolve the issues.

8.2 Final Report

The Contractor shall submit a final report. The report shall be a narrative report, in MS Word format and shall include data to support all conclusions drawn.

The final report shall:

(1) Detail the methodology, including study design, sampling strategy, and data collection methods, including a summary of each task;

(2) Quantify the number of survey participants and respondents;

(3) Discuss the analysis performed and outcomes realized consistent with Task 6;

(4) Provide a summary of the comparison performed in Task 7, including the organizations/agencies utilized for the comparison, specific areas of comparison, and outcomes realized;

(5) Conclusions. Based upon all data collected, discuss how the SURF Program is beneficial to NIST, whether or not it continues to inspire undergraduate students to pursue careers in STEM through a unique research experience that supports the NIST mission, and any areas whereas the Program can be improved.

The final report shall include information pertinent to the assessment goals to document whether the program is beneficial to NIST and whether it continues to meet stated program goals. This document shall be a narrative report and include a summary of the supporting data to support any conclusions drawn. It shall detail the methodology, including study design, sampling strategy, and data collection methods, including a summary of each section or milestone section.

8.3 Presentation

The Contractor shall summarize and present the final report to select NIST management and staff. The Contractor shall prepare all presentation materials and provide the materials to the COR for review and approval not later than five days prior to the scheduled presentation date. Changes must be made in accordance with COR comments. The presentation shall be virtual. The NIST COR shall schedule the meeting and provide the web link for the meeting.

IV. DELIVERABLES

0001/2 Evaluation Methodology 1 30 days after Contractor Personnel are vetted as NIST Associates

0001/4 Execute Survey Upon receipt of OMB Clearance

0001/6 Electronic File with all Survey Responses 60 days after survey closes

0001/7 Progress Reports 4 Every 3 months

0001/7 Final Report 1 11 months after award

0001/7 Presentation 1 2-3 weeks after approval of final report

OPTION PERIOD I

0001B/1 Kick-Off Meeting 1 15 days after exercise of option

0001B/4 Execute Survey Upon approval by OMB (if needed)

0001C/5 Student Information Follow-Up If Exercised

0001B/6 Electronic File with all 60 days after survey closes Survey Responses

0001B/7 Progress Reports 4 Every 3 months

0001B/7 Final Report 1 11 months after option exercise

0001B/7 Presentation 1 2-3 weeks after approval of final report

GOVERNMENT FURNISHED PROPERTY, DATA AND/OR INFORMATION

The Government will provide data on the program and previous participants between 2012-2022.

All data and information provided by the Government during the performance of this project remains the property of the Government and shall be surrendered to the Government upon completion or termination of this requirement. Likewise, all deliverables generated under this project remain the property of the Government. To achieve the objectives of the project, the Contractor is expected to use available data, as well as collect additional information to compile data on which to base its assessment.

Available sources of data will include, but not be limited to:

a. Program lists of previous SURF students

b. Existing SURF program data

c. Previous surveys of former SURF participants

d. Preliminary logic model for the program

Contractor personnel must be fully vetted as NIST Associates prior to obtaining access to any of the above referenced data sources.

PERIOD OF PERFORMANCE

The period of performance shall be one year from the date of award with one 1-year option period.

PERSONNEL REQUIREMENTS

Key Personnel:

The Government will identify the Project Manager as Key Personnel in the resultant contract. Key Personnel are defined in the Department of Commerce Acquisition Regulation (CAR) Clause 1352.237-75 KEY PERSONNEL, incorporated in full text herein.

Project Manager The Project Manager shall be the primary liaison between the Contractor and the COR. The Project Manager must meet all of the following requirements:

a. Ph.D. or Ed.D;

b. Minimum of five years of experience in survey methodology and program evaluations;

c. Experience evaluating STEM workforce development programs established at U.S. federal laboratories.

Additional Personnel The Contractor must have the resources to establish and maintain a technical team, throughout the life of the contract, who, collectively possess experience evaluating STEM workforce development programs established at U.S. federal laboratories.