Attachment A_PWS 1333ND25QNB770146.pdf

Solicitation Number 1333ND25QNB770146

Attachment A Performance Work Statement (PWS)

Title: Computer-Based Testing for the Cryptographic Validation Program (CVP)

I. Background The National Institute of Standards and Technology (NIST) promotes the U.S. economy and public welfare by providing technical leadership for the nation’s measurement and standards infrastructure. The Cryptographic Module Validation Program (CMVP) was developed to support the federal user communities for strong, independently tested, and commercially available cryptographic modules. Through this program, the CMVP works with international government, public and private sectors as a part of the cryptographic community to achieve standards-based security and assurance of correct implementation. Federal agencies are required to use validated cryptographic modules for the protection of sensitive unclassified information.

The goal is to provide those agencies, as well as other users, with a security metric list to use in procuring and deploying validated cryptographic modules. In other words, all non-DOD Federal Agencies must use software that has been validated by the CMVP when it comes to security related to cryptographic modules.

In advancing its mission objectives, NIST must validate cryptographic modules and cryptographic algorithms, and ensure independent, National Voluntary Laboratory Accreditation Program (NVLAP) accredited laboratories meet competence and performance criteria for conducting algorithm and module testing. Cryptographic and Security Testing (CST) Laboratories are independent laboratories accredited by NVLAP. CST Labs verify each module meets a set of testable cryptographic and security requirements, with each CST laboratory submission reviewed and validated by CMVP. These labs are located around the world, including North America, Asia, Australia, and Europe.

To maintain quality in the CMVP an exam is given to all testers. They must pass this exam to evaluate and test the modules. Currently the exam is 100 questions which the CMVP has developed over many years. The exam cannot be copied or removed from the test site so the questions will not become public knowledge. In addition, strict ID verification must be conducted to ensure the test takers are who they say they are.

II. Purpose The purpose of this requirement is to publish, administer, and update computer-based competency tests to testing centers near NVLAP accredited laboratories.

Objective:

NIST’s Computer Security Division’s (CSD) objective for this tasking includes administering computer-based competency exams to NVLAP accredited laboratory testers for the Security Testing, Validation and Measurement (STVM) Group’s cryptographic validation programs.

• A secure location for taking the exam

• ID verification capabilities

• On-line registration capabilities

• Administering computer-based competency exams to NVLAP accredited laboratory testers for the Security Testing, Validation and Measurement (STVM) Group’s cryptographic validation programs

• Test centers to be located within a reasonable distance from the labs

• The capability to add new testing locations as new CST Labs come online

Period of Performance The period of performance shall be a base period of twelve (12) months plus one (1) 12-month option period. The estimated start date is May 1, 2025. The exact dates will be entered at time of award.

Place of Performance The place of performance shall be at the contractor’s facility.

III. Specific Requirements

Tasks The contractor shall provide all labor, project oversight, administration and technical execution of the tasks and deliverables identified in this Performance Work Statement (PWS). The contractor shall be responsible for maintaining accurate records of project activities and shall provide the support services described below.

1.1 Test Publishing and Administration (Base and Option Period) The contractor shall:

1. Prepare/update candidate instructions that are appropriate for NIST to distribute to the NVLAP accredited laboratories. The instructions shall be specific to the NIST test. For example, the information may include the following:

a. Test duration

b. Materials that may be brought to testing facility

c. Versions of documents that will be available

d. How to register

2. Publish and administer the FIPS 140-3 competency exam via a computer-based system.

This shall include the following:

a. Provide supporting documents to test candidates during administration of the exam. On a periodic schedule, NIST will supply contractor with documents that are authorized for use during the exam

b. Contractor shall also provide a means for candidates to address issues/questions during the test

3. Provide evidence of secure testing centers worldwide, of which 85% shall be within 50 miles of NVLAP accredited laboratories currently located in the US, Canada, Japan, Germany, Malaysia, Spain, France, and Taiwan. A current list of NVLAP accredited labs may be found in Appendix A at the end of this document. More CST labs may be added during this contract period which will increase the number of testing locations needed.

NIST will notify the contractor in order to request a new test center

4. Schedule candidate testing

5. Contractor shall collect a $400 fee from candidates. It is anticipated that 30 to 40 candidates will require taking the exam per year

6. Verify candidates with NIST to ensure candidates are eligible to sit for the test

7. Verify/authenticate candidate’s identity

8. Protect exam content from unauthorized removal by monitoring candidates during the entirety of the exam

9. Secure candidates test results and share those results only with NIST

10. NIST will notify candidate of results

1.2 Exam Maintenance (Option Period Only)

The contractor shall:

1. Work collaboratively with NIST to review the FIPS 140-3 competency exam and update up to 10% of the test questions, of each exam, once a year. Updated raw questions will be provided to the contractor by NIST, but will require:

a. Vetting new questions for bias and difficulty and updating scoring appropriately

b. Removing and/or editing questions

c. Balancing and assessing questions

d. Updating tests to incorporate changes

1.3 Project Management & Reporting

1.3.1 Kick-off Meeting (Base Period Only)

The contractor shall attend a virtual kick-off meeting within 10 business days of award where the TO PM will identify Key Personnel and ask the government any questions they may have at that time.

1.3.2 Project Reporting (Base and Option Period)

The Contractor shall provide a monthly report, via email, of the status of the project to the government Technical Point of Contact (TPOC) and Contracting Officer’s Representative (COR), and other stakeholders identified by NIST. The monthly status reports shall detail activities accomplished, deliverables completed, outstanding deliverables, any delays, reasons for delays and proposed resolutions, and recommendations.

Figure 1: Government Work Break Down Structure (WBS)

CVP Computer-Based Testing

Base Period

4.1 Test Publishing & Administration

4.2 Exam Maintenance

4.3 Project Management & Reporting

4.3.1 Kick-off Meeting

4.3.2 Project Reporting

Option Period 1

4.1 Test Publishing & Administration

4.2 Exam Maintenance

4.3 Project Management & Reporting 4.3.2 Project Reporting

IV. General Requirements

Government-Furnished Property, Material, Equipment, or Information (GFP, GFM, GFE, or GFI)

No Government-furnished property or equipment will be required as a part of this effort.

The government will furnish information as needed to develop and update exams. The Government will provide electronic copies of any relevant documents necessary to complete the tasks.

Key Personnel

It is anticipated that one (1) Contractor Key Personnel be required and shall meet the following minimum qualifications.

• Computer-Based Test Analyst (or equivalent) and shall have 2 years of experience in developing computer-based test questions from questions supplied by the customer.

V. Schedule of Deliverables and Performance Requirements Summary (PRS)

All deliverables shall be delivered to NIST via secure methods as directed by the Technical Lead or the COR. The COR will evaluate the deliverables for completeness and will either accept or reject within 10 days of contractor submission. All deliverables shall be provided to the COR.

Description Projected Completion Date Media / Desired Output / Performance Standard Monitoring

Method Base and Option Period 1

D1 4.1.1 Instructions for candidates prepared/updated

Delivery date within 2 months of award

• MS Outlook / MS Word / Contractor format

• Candidate instructions that NIST will distribute to the

NVLAP laboratories that are specific to the NIST test

• Instructions shall be free of grammatical and typographical errors

TPOC and COR Review

Description Projected Completion Date Media / Desired Output / Performance Standard Monitoring

Method Base and Option Period 1

• Information may include, but not be limited to, the following:

Test duration Materials that may be brought to testing facility Versions of documents that will be available How to register

D2 4.1.2

FIPS 140-3 Competency exam published & administered via a computer-based system

Test administration to begin immediately after Base award or award of modification

• Contractor format

• Competency exam shall consist of two 100 question exams, of which one will be chosen at the time of testing to be administered via a computer-based system

• Publish and administer exam as developed in collaboration with the NIST technical team

• Provide NIST supplied supporting document to test candidates during administration of the exam Provide a means for candidates to address issues/questions during the exam

TPOC and COR Review

D3 4.1.3 Evidence of security measures taken at testing centers

To begin immediately after Base award or award of modification

• MS Outlook / MS Word / Contractor format during monthly status reports at a minimum

• Evidence of a secure testing environment

• Emails or reports documenting security measures that are taken at each testing facility to prevent unapproved persons from taking the test as well as to prevent the theft of text questions/materials

TPOC and COR Review

D4 4.1.4 Evidence of candidate test scheduling

To begin immediately after Base award or award of modification

• MS Outlook / MS Word / Contractor format during monthly status reports at a minimum

• Monthly report of upcoming scheduled tests emailed to COR & TPOC on first of each month and available within 2 business days upon request if there are questions/problems with a candidate getting scheduled

Description Projected Completion Date Media / Desired Output / Performance Standard Monitoring

Method Base and Option Period 1

D5 4.1.6 Candidate test eligibility verified

To begin immediately after Base award or award of modification

• MS Outlook / MS Word

• Notification from contractor, via report or email, that a candidate wishes to schedule a test

• TPOC verifies candidate’s eligibility and informs contractor whether candidate is eligible to take the test

TPOC and COR Review

D6 4.1.7 Candidate identity verified/ authenticated

To begin immediately after Base award or award of modification

• MS Outlook / MS Word / Contractor format

• Report of measures taken to authenticate people before the test is administered so that only verified candidates are tested

• Records for each verified candidates shall be kept on file should case problems arise

TPOC and COR Review

D7 4.1.8 Exam content protected by monitoring candidates for entirety of exam

To begin immediately after Base award or award of modification

• MS Outlook / MS Word / Contractor format

• Secure exam content so that it is not replicated outside the testing environment by laboratories

• A written record per applicant verifying that the applicant was monitored for the entirety of the test and there was no observation of the candidate copying or otherwise duplicating the exam material

TPOC and COR Review

D8 4.1.9 Candidate test results secured and shared only with NIST

To begin immediately after Base award or award of modification

• MS Outlook / MS Word / Contractor format

• Tests are calculated and results are sent to NIST for each candidate

• A written record of statistics from the exam for a candidate

• Contractor shall not share results with the candidate, they shall share the results with only NIST and NIST shall inform the lab/candidate of their score and results

TPOC and COR Review

D9 4.2 FIPS 140-3 competency exam reviewed & updated up to 10% each exam

Once a year for each Option Period

• Contractor’s format

• Update of exam shall include:

Review test and update up to 10% of test questions, Description Projected Completion Date Media / Desired Output / Performance Standard Monitoring

Method Base and Option Period 1 twice a year Updated raw questions will be provided by NIST but will require:

Vetting new questions for bias and difficulty and updating scoring appropriately Removing and/or editing questions Balancing and assessing the difficulty of each question in order to assist NIST with evaluating questions

Updating test to incorporate changes Assembling a balanced test from a pool of questions or multiple tests created that are managed by the vendor in order to prevent candidates and/or laboratories from memorizing

Determination of passing score Working with NIST to assess post test score patterns and/or issues Editing to be free of grammatical and typographical errors

• Initial delivery within 6 months of award, then twice yearly

D1 0 4.3.1 Kick-off Meeting (Base

Period only) Within 10 business days after award

• MS Word / MS Excel / MS PowerPoint

• The Kick-off meeting shall be no later than 10 business days after Base Period award

• Electronic copy of the agenda shall be delivered to the

COR at least 2 business days before the meeting

• The kick-off meeting shall be utilized to introduce all members of the contractor’s team, review all requirements, deliverables, and schedule.

Description Projected Completion Date Media / Desired Output / Performance Standard Monitoring

Method Base and Option Period 1

D1 1 4.3.2 Project Reporting Monthly

• MS Word / MS Excel / MS PowerPoint submitted via email

• Monthly updates shall be submitted to the COR before each monthly meeting

• Reports shall be legible and of a professional quality

• Any problems with the work, current or anticipated, shall be clearly reported

Appendix A Current list of Laboratories

Current list of National Voluntary Laboratory Accreditation Program (NVLAP) laboratories obtained from the CSRC website: https://www-s.nist.gov/niws/index.cfm?event=directory.search#no-back

Lab Name City State Country

Acumen Security Rockville MD US

ADVANCED DATA SECURITY San Jose CA US

AEGISOLVE, Inc. Mountain View CA US

Apple Inc. SECLAB Cupertino CA US

Asia Pacific IT Laboratory, TUV NORD Kaohsiung TW

ATSEC information security corporation Austin TX US

Booz Allen Hamilton Cyber Assurance Testing Laboratory

Laurel MD US

Cisco Systems Automated Cryptographic Validation Protocol Lab

Morrisville NC US

CyberSecurity Malaysia Cryptographic Evaluation Laboratory

Selangor MY

DEKRA Cybersecurity Certification Laboratory

Sterling VA US

Dekra Testing and Certification S.A.U. San Sebastian de los Reyes, Madrid

ES

ECSEC Laboratory Inc. Tokyo JP

EWA – Canada Ottawa, ON CA

Google LLC Mountain View CA US

Gossamer Security Solutions Columbia MD US

Intel CST Lab Hillsboro OR US

IT Security Center Chiyoda-ku, Tokyo JP https://www-s.nist.gov/niws/index.cfm?event=directory.search#no-back https://www-s.nist.gov/niws/index.cfm?event=directory.search#no-back

Current list of Laboratories

Leidos Accredited Testing & Evaluation (AT&E) Lab

Columbia MD US

Lightship Security Inc. Ottawa, Ontario CA

NXP ACVP Laboratory Toulouse FR

Penumbra Security, Inc. Clackamas OR US

SERMA SAFETY AND SECURITY Pessac Cedex FR

Teron Labs Deakin AU

TUVIT Evaluation Body for IT Security Essen DE

UL Verification Services, Inc. San Luis Obispo CA US