Performance Work Statement_CVP.pdf

Computer-Based Testing for the Cryptographic Validation Program (CVP)

Performance Work Statement

1.0 Background

The National Institute of Standards and Technology (NIST) promotes the U.S. economy and public welfare by providing technical leadership for the nation’s measurement and standards infrastructure. The Cryptographic Module Validation Program (CMVP) was developed to support the federal user communities for strong, independently tested, and commercially available cryptographic modules. Through this program, the CMVP works with international government, public and private sectors as a part of the cryptographic community to achieve standards-based security and assurance of correct implementation. Federal agencies are required to use validated cryptographic modules for the protection of sensitive unclassified information. The goal is to provide those agencies, as well as other users, with a security metric list to use in procuring and deploying validated cryptographic modules. In other words, all non- DOD Federal Agencies must use software that has been validated by the CMVP when it comes to security related to cryptographic modules.

In advancing its mission objectives, NIST must validate cryptographic modules and cryptographic algorithms, and ensure independent, National Voluntary Laboratory Accreditation Program (NVLAP) accredited laboratories meet competence and performance criteria for conducting algorithm and module testing. Cryptographic and Security Testing (CST) Laboratories are independent laboratories accredited by NVLAP. CST Labs verify each module meets a set of testable cryptographic and security requirements, with each CST laboratory submission reviewed and validated by CMVP. These labs are located around the world, including North America, Asia, Australia, and Europe.

To maintain quality in the CMVP an exam is given to all testers. They must pass this exam to evaluate and test the modules. Currently the exam is 100 questions which the CMVP has developed over many years. The exam cannot be copied or removed from the test site so the questions will not become public knowledge. In addition, strict identity verification must be conducted to ensure the test takers are who they say they are.

2.0 Objectives

NIST’s Computer Security Division’s (CSD) objective for this tasking includes: administering computer-based competency exams to NVLAP accredited laboratory testers for the Security Testing, Validation and Measurement (STVM) Group’s cryptographic validation programs.

• A secure location for taking the exam

• Identity (ID) verification capabilities

• On-line registration capabilities

• Administering in-person computer-based competency exams to NVLAP accredited laboratory testers for the Security Testing, Validation and Measurement (STVM) Group’s cryptographic validation programs

• Test centers to be located within a reasonable distance from the labs

• The capability to add new testing locations as new CST Labs come online

3.0 Scope

The contractor shall develop, publish, update, and administer computer-based competency tests to testing centers near NVLAP accredited laboratories.

This is a firm-fixed price (FFP) contract requirement.

4.0 Tasks

The contractor shall provide all labor, project oversight, administration and technical execution of the tasks and deliverables identified in this Performance Work Statement (PWS). The contractor shall be responsible for maintaining accurate records of project activities and shall provide the support services described below.

4.1 Test Development (Base Period)

The contractor shall:

1. Work collaboratively with NIST to develop a competency exam for FIPS 140-3 based on questions and answers developed by NIST, to be administered via a computer-based system. The development of the exam shall include the following:

a. Editing and balancing the questions as necessary to remove bias and ensure correct grammar

b. Assessing the difficulty of each question in order to assist NIST with assigning weight for scoring

c. Assembling a balanced test from the pool of questions or create multiple tests that the vendor can manage in order to prevent candidates/laboratories from memorizing the test(s)

d. Determining a passing score

e. Provide data to NIST to assess post test score patterns and/or issues

2. Prepare/update candidate instructions that are appropriate for NIST to distribute to the NVLAP accredited laboratories. The instructions shall be specific to the NIST test. For example, the information may include, but not be limited to, the following:

a. Test duration

b. Materials that may be brought to testing facility

c. Versions of documents that will be available

d. How to register

4.2 Test Publishing and Administration (Base and Option Periods)

The contractor shall:

1. Publish and administer the FIPS 140-3 competency exam via a computer-based system.

This shall include the following:

a. Provide supporting documents to test candidates during administration of the exam. On a periodic schedule, NIST will supply contractor with documents that are authorized for use during the exam

b. Contractor shall also provide a means for candidates to address issues/questions during the test

2. Provide evidence of secure testing centers worldwide, of which 85% shall be within 50 miles of NVLAP accredited laboratories currently located in the US, Canada, Japan, Germany, Malaysia, Spain, France, and Taiwan. A current list of NVLAP accredited labs may be found in Appendix A at the end of this document. More CST labs may be added during this contract period which will increase the number of testing locations needed.

NIST will notify the contractor in order to request a new test center

3. Schedule candidate testing

4. Contractor shall collect a $400 fee from candidates. It is anticipated that 30 to 40 candidates will require taking the exam per year

5. Verify candidates with NIST to ensure candidates are eligible to sit for the test

6. Verify/authenticate candidate’s identity

7. Protect exam content from unauthorized removal by monitoring candidates during the entirety of the exam

8. Secure candidates test results and share those results only with NIST

9. NIST will notify candidate of results

4.3 Planning and Reporting

4.3.1 Kick-off Meeting (Base Period)

The contractor shall attend one kick-off meeting, held virtually due to COVID-19 restrictions, no later than 10 business days after award of the order. The kick-off meeting shall be utilized to introduce all members of the contractor’s team and review all requirements, deliverables and project scheduling as applicable.

4.3.2 Regular status and reporting (Base and Option Periods)

The Contractor shall provide a monthly report via email of the status of the project to the government Technical Point of Contact (TPOC) and Contracting Officer’s Representative (COR), and other stakeholders identified by NIST. The monthly status reports shall detail activities accomplished, deliverables completed, outstanding deliverables, any delays, reasons for delays and proposed resolutions, and recommendations. At the discretion of NIST the contractor may be asked to participate in ad-hoc virtual status update meetings.

4.4 Exam Maintenance (Option Periods)

The contractor shall:

1. Work collaboratively with NIST to review the FIPS 140-3 competency exam and update up to 10% of the test questions twice a year. Updated raw questions will be provided to the contractor by NIST, but will require:

a. Vetting new questions for bias and difficulty and updating scoring appropriately

b. Removing and/or editing questions

c. Balancing and assessing questions

d. Updating tests to incorporate changes

Figure 1: Government Work Break Down Structure (WBS)

CVP Computer-Based Testing

Base Period

4.1 Test Development

4.2 Test Publishing &

Administration

4.3 Planning & Reporting

4.3.1 IPWP

4.3.2 Kick-off Meeting

4.3.3 Status Reporting

Option Period 1

4.2 Test Publishing &

Administration

4.3 Planning &

Reporting

4.3.1 IPWP

4.3.3 Status Reporting

4.4 Exam Maintenance

Option Period 2

4.2 Test Publishing &

Administration

4.3 Planning &

Reporting

4.3.1 IPWP

4.3.3 Status Reporting

4.4 Exam Maintenance

5.0 Deliverables, Due Dates and Performance Requirements Summary (PRS)

All deliverables shall be delivered to NIST via secure methods as directed by the Technical Lead or the COR. The COR will evaluate the deliverables for completeness and will either accept or reject within 10 days of contractor submission. All deliverables shall be provided to the COR.

Description Projected Completion Date Media / Desired Output / Performance Standard Monitoring

Method Base Period

D1 4.1 FIPS 140-3 Competency exam developed

Delivery date within 6 months of award

• Contractor format

• NIST will create a minimum of 200 questions which will result in two 100 question exams

• Development of the exam shall include:

Editing to be free of grammatical and typographical errors Editing and balancing of questions to remove bias Assessing the difficulty of each question in order to assist NIST with assigning weight for scoring Assembling a balanced test from a pool of questions or multiple tests created that are managed by the vendor in order to prevent candidates and/or laboratories from memorizing the test(s)

Determination of passing score Working with NIST to assess post test score patterns and/or issues

TPOC and COR Review

D2 4.1 Instructions for candidates prepared/updated

Delivery date within 6 months of award

• MS Outlook / MS Word / Contractor format

• Candidate instructions that NIST will distribute to the

NVLAP laboratories that are specific to the NIST test

• Instructions shall be free of grammatical and typographical errors

TPOC and COR Review

• Information may include, but not be limited to, the following:

Test duration Materials that may be brought to testing facility Versions of documents that will be available How to register

D3 4.2

FIPS 140-3 Competency exam published & delivered via a computer-based system

Test administration to begin the 7th month after award

• Contractor format

• Competency exam shall consist of two 100 question exams, of which one will be chosen at the time of testing to be administered via a computer-based system

• Publish and deliver exam as developed in collaboration with the NIST technical team

• Provide NIST supplied supporting document to test candidates during administration of the exam

• Provide a means for candidates to address issues/questions during the exam

TPOC and COR Review

D4 4.2 Evidence of security measures taken at testing centers

To be completed throughout the base period

• MS Outlook / MS Word / Contractor format during monthly status reports at a minimum

• Evidence of a secure testing environment

• Emails or reports documenting security measures that are taken at each testing facility in order to prevent unapproved persons from taking the test as well as to prevent the theft of text questions/materials

TPOC and COR Review

D5 4.2 Evidence of candidate test scheduling

To be completed throughout the base period

• MS Outlook / MS Work / Contractor format during monthly status reports at a minimum

• Monthly report of upcoming scheduled tests emailed to COR & TPOC on first of each month and available within 2 business days upon request if there are questions/problems with a candidate getting scheduled

D6 4.2 Candidate test eligibility verified

To be completed throughout the base period

• MS Outlook / MS Word

• Notification from contractor, via report or email, that a candidate wishes to schedule a test

• TPOC verifies candidate’s eligibility and informs contractor whether or not candidate is eligible to take the test

TPOC and COR Review

D7 4.2 Candidate identity verified/ authenticated

To be completed throughout the base period

• MS Outlook / MS Word / Contractor format

• Report of measures taken to authenticate people before the test is administered so that only verified candidates are tested

• Records for each verified candidates shall be kept on file should case problems arise

TPOC and COR Review

D8 4.2 Exam content protected by monitoring candidates for entirety of exam

To be completed throughout the base period

• MS Outlook / MS Word / Contractor format

• Secure exam content so that it is not replicated outside the testing environment by laboratories

• A written record per applicant verifying that the applicant was monitored for the entirety of the test and there was no observation of the candidate copying or otherwise duplicating the exam material

TPOC and COR Review

D9 4.2 Candidate test results secured and shared only with NIST

To be completed throughout the base period

• MS Outlook / MS Word / Contractor format

• Tests are calculated and results are sent to NIST for each candidate

• A written record of statistics from the exam for a candidate

• Contractor shall not share results with the candidate, they shall share the results with only NIST and NIST shall inform the lab/candidate of their score and results

TPOC and COR Review

D11 4.3.2 Kick-off Meeting Within 10 business days after award • MS Word / MS Excel / MS PowerPoint

• The Kick-off meeting shall be no later than 10 business days after award

• Electronic copy of the agenda shall be delivered to the COR at least 2 business days before the meeting

• The kick-off meeting shall be utilized to introduce all members of the contractor’s team, review all requirements, deliverables, and schedule.

D12 4.3.3 Status Reporting Monthly

• MS Word / MS Excel / MS PowerPoint submitted via email

• Monthly updates shall be submitted to the COR before each monthly meeting

• Reports shall be legible and of a professional quality

• Any problems with the work, current or anticipated, shall be clearly reported

TPOC and COR Review

Description Projected Completion Date Media / Desired Output / Performance Standard Monitoring

Method Option Periods 1 & 2

D13 4.2

FIPS 140-3 Competency exam published & delivered via a computer-based system

On-going throughout option period(s) from the start

• Contractor format

• Competency exam shall consist of two 100 question forms, of which one will be chosen at the time of testing to be administered via a computer-based system

• Publish and deliver exam as developed in collaboration with the NIST technical team

• Provide NIST supplied supporting document to test candidates during administration of the exam

• Provide a means for candidates to address issues/questions during the exam

D14 4.2 Evidence of security measures taken at testing centers

To be completed throughout the option periods

• MS Outlook / MS Word / Contractor format documented in monthly reports at a minimum

• Evidence of a secure testing environment

• Emails or reports documenting security measures that are taken at each testing facility in order to prevent unapproved persons from taking the test as well as to prevent the theft of text questions/materials

TPOC and COR Review

D15 4.2 Evidence of candidate test scheduling

To be completed throughout the option periods

• MS Outlook / MS Work / Contractor format documented in monthly reports at a minimum

• Monthly report of upcoming scheduled tests emailed to COR & TPOC on first of each month and available within 2 business days upon request if there are questions/problems with a candidate getting scheduled

TPOC and COR Review

D16 4.2 Candidate test eligibility verified

To be completed throughout the option periods

• MS Outlook / MS Word

• Notification from contractor, via report or email, that a candidate wishes to schedule a test

• TPOC verifies candidate’s eligibility and informs contractor whether or not candidate is eligible to take the test

TPOC and COR Review

D17 4.2 Candidate identity verified/ authenticated

To be completed throughout the option periods

• MS Outlook / MS Word / Contractor format

• Report of measures taken to authenticate people before the test is administered so that only verified candidates are tested

• Records for each verified candidates shall be kept on file should case problems arise

TPOC and COR Review

D18 4.2 Exam content protected by monitoring candidates for entirety of exam

To be completed throughout the option periods

• MS Outlook / MS Word / Contractor format

• Secure exam content so that it is not replicated outside the testing environment by laboratories

• A written record per applicant verifying that the applicant was monitored for the entirety of the test and there was no observation of the candidate copying or otherwise duplicating the exam material

D19 4.2 Candidate test results secured and shared only with NIST

To be completed throughout the option periods

• MS Outlook / MS Word / Contractor format

• Tests are calculated and results are sent to NIST for each candidate

• A written record of statistics from the exam for a candidate

• Contractor shall not share results with the candidate, they shall share the results with only NIST and NIST shall inform the lab/candidate of their score and results

TPOC and COR Review

D20 4.3 FIPS 140-3 competency exam reviewed & updated up to 10%

Twice a year for each option period

• Update of exam shall include:

Review test and update up to 10% of test questions, twice a year Updated raw questions will be provided by NIST but will require:

Vetting new questions for bias and difficulty and updating scoring appropriately Removing and/or editing questions Balancing and assessing the difficulty of each question in order to assist NIST with evaluating questions

Updating test to incorporate changes Assembling a balanced test from a pool of questions or multiple tests created that are managed by the vendor in order to prevent candidates and/or laboratories from memorizing

Determination of passing score Working with NIST to assess post test score patterns and/or issues Editing to be free of grammatical and typographical errors

• Initial delivery within 6 months of award, then twice yearly

D21 4.3.3 Status Reporting Monthly

• MS Word / MS Project / MS PowerPoint via email

• Monthly updates shall be submitted to the COR before each monthly meeting

• Reports shall be legible and of a professional quality

• Any problems with the work, current or anticipated, shall be clearly reported

6.0 Government-Furnished Property, Material, Equipment, or Information (GFP, GFM, GFE, or GFI)

No Government-furnished equipment will be used as a part of this effort.

The government will furnish information as needed to develop and update exams. The Government will provide electronic copies of any relevant documents necessary to complete the tasks.

7.0 Key Personnel

One (1) Contractor Key Personnel is required and shall meet the following minimum qualifications. The title of the labor category given below is not mandatory. It is simply an example of a title suitable to the type of work to be performed:

• Computer-Based Test Analyst: Shall have 2 years of experience in developing computer-based test questions from questions supplied by the customer.

8.0 Place of Performance

All tasks shall be performed at the contractor’s facility and meetings will be scheduled virtually as needed.

9.0 Period of Performance

The award shall have a 12-month base period of performance as well as two (2) 12-month option periods which may or may not be exercised by the Government in accordance with FAR 52.217-9: Option to Extend the Term of the Contract.

Appendix A Current list of Laboratories

Current list of National Voluntary Laboratory Accreditation Program (NVLAP) laboratories obtained from the CSRC website: https://www-s.nist.gov/niws/index.cfm?event=directory.search#no-back

Lab Name City State Country

Acumen Security Rockville MD US

ADVANCED DATA SECURITY San Jose CA US

AEGISOLVE, Inc. Mountain View CA US

Asia Pacific IT Laboratory, TUV NORD Kaohsiung City TW

ATSEC information security corporation Austin TX US

Booz Allen Hamilton Cyber Assurance Testing Laboratory

Laurel MD US

CyberSecurity Malaysia Cryptographic Evaluation Laboratory

Cyberjaya, Selangor MY

Dekra Testing and Certification S.A.U. San Sebastian de los Reyes, Madrid

ES

ECSEC Laboratory Inc. Tokyo JP

EWA – Canada Ottawa, ON CA

Gossamer Security Solutions Columbia MD US

IT Security Center Chiyoda-ku, Tokyo JP

Leidos Accredited Testing & Evaluation (AT&E) Lab

Columbia MD US

Lightship Security Inc. Lightship Security Inc. CA

Penumbra Security, Inc. Clackamas OR US

Serma Technologies ITSEF Pessac FR

TÜV Evaluation Body for IT-Security Essen DE

UL Verification Services, Inc. San Luis Obispo CA US https://www-s.nist.gov/niws/index.cfm?event=directory.search#no-back https://www-s.nist.gov/niws/index.cfm?event=directory.search#no-back

4.4 Exam Maintenance (Option Periods)