RFP Attachment 4 - Task Order 0001 PWS.pdf

ID09210029 - RFP 47QFPA21R0010 Attachment 4

General Services Administration (GSA) Federal Acquisition Service (FAS)

Assisted Acquisition Services (AAS) Region 9

Contract # TBD

COI # ID09210029

MPTA Task Order #0001

Performance Work Statement

1.0 TITLE

Psychometric Support for the ASVAB – Form Development.

2.0 BACKGROUND

The Armed Services Vocational Aptitude Battery (ASVAB) is a multiple aptitude test battery used by all military Services to select and classify military applicants. The ASVAB is currently administered as both a paper-and-pencil (P&P) test and a computer adaptive test (CAT), although eventual elimination of P&P testing is a goal of the program. Hence, the development of new test forms/item pools is currently targeted mainly toward CAT-ASVAB, with occasional development of P&P-ASVAB forms, on an as needed basis. [Note that under ASVAB terminology, both P&P test forms and CAT item pools are referred to as “forms.”]

3.0 SCOPE

The objective of this effort is to develop the subsequent CAT-ASVAB forms for use in the Enlistment Testing Program and the Career Exploration Program. The steps to developing these numbers of CAT-ASVAB forms include cleaning and scoring data, conducting item analyses and screening items, assembling item pools, and equating test scores across new and existing CAT-ASVAB item pools.

4.0 REQUIREMENTS

4.1 CAT-ASVAB Form Development

4.1.1 Data Management. The contractor shall process and clean CAT-ASVAB data for purposes such as eliminating duplicate records, eliminating corrupt records, eliminating records with invalid responses, and retaining one random record for repeat testers. The contractor shall score the item

Assisted Acquisition Services (AAS) Region 9 responses and format the CAT-ASVAB data for use in activities such as item calibrations, parameter rescalings, and Differential Item Functioning (DIF) analyses as an initial assessment of measurement bias, and sensitivity reviews. The contractor shall also work with OPA staff to verify/finalize item keys prior to scoring the response data and conducting item calibrations.

4.1.2 Item Calibrations and Parameter Rescalings. The contractor shall conduct item calibrations using calibration software designated by OPA.

The contractor shall rescale the resulting item parameters to be on the operational CAT-ASVAB scale, using an approach designated by OPA.

4.1.3 Item Analyses and Screenings. The contractor shall compute item analyses specified by OPA, including classical item statistics, distractor analyses, DIF analyses, Item Response Theory (IRT) item information functions, and IRT item fit statistics. The contractor shall make keep/drop decisions for all items based on classical and IRT-based item analyses. The contractor shall also work with OPA to identify items flagged for non-statistical reasons that will be excluded from the analyses (i.e., sensitivity reviews as mentioned in paragraph 4.1.1), and to update OPA’s item history files to reflect keep/drop decisions.

4.1.4 Create Enemy Groups. The contractor shall identify enemies and create enemy groupings for each ASVAB subtest prior to building final new forms/pools. The contractor shall work with DMDC to develop extensive enemy groupings for the Mathematics Knowledge and Mechanical Comprehension subtests, based on content evaluations/comparisons.

Enemy groupings for the other ASVAB subtests will be based on documentation provided by OPA..

4.1.5 Build Forms/Pools. The contractor shall build preliminary and final parallel test forms by subtest from a set of available items identified by OPA. Final pools will take into account item enemies. The contractor shall evaluate the item pools by conducting simulations and computing score information functions, as well as comparing the precision to appropriate CAT-ASVAB forms. Final evaluations of item pools will also entail a key check, plus evaluations of content coverage and within form item enemies.

Assisted Acquisition Services (AAS) Region 9

4.1.6 Conduct Preliminary Equating. The contractor shall conduct a preliminary theoretical equating of the new adaptive test forms to the appropriate operational and reference CAT-ASVAB test forms. The preliminary equating will be used during an Initial Operational Test and Evaluation of the test forms, which will provide the data for a final equating.

4.1.7 Conduct Final Equating. The contractor shall conduct a final empirical equating of the new adaptive test forms using data collected in a random groups equating design. This equating will be used to score the exams of applicants tested subsequent to the final equating implementation.

4.1.8 Procedure Refinement. Refine statistical and analytic form development procedures. These refined procedures will provide more accurate and efficient methods for data management, item calibration and scaling, item screening, enemy group construction, pool assembly, impact of keeping/dropping misfitting items, and equating.

4.1.9 Contractor Manpower Reporting. If required for this task order after resolution of DFARS case 2018-D063, the contractor shall report contractor labor hours (including subcontractor labor hours) required for performance of services provided under this contract and associated data via a secure collection method, through www.sam.gov. Reporting inputs will be for the labor executed during the POP during each Government fiscal year (FY), which runs October 1 through September 30.

5.0 DELIVERABLES AND REPORTING REQUIREMENTS

5.1 Monthly Status Reports. The contractor shall document the status of the effort each month

5.2 Technical Documents. The contractor shall provide written documentation of steps taken and outcomes for each task, such as you would find in a technical manual or technical bulletin.

5.3 Data Files. The contractor shall provide copies of all data files/datasets that are created and/or used in the analyses.

Assisted Acquisition Services (AAS) Region 9

5.4 Source Code. The contractor shall provide written documentation of all computer programs, to include flow charts of all main and other routines.

System documentation shall include electronic versions of computer programs.

Computer source code shall be delivered in electronic format.

5.5 Schedule of Deliverables.

Deliverable Due Date

Monthly Progress Report Monthly, starting with the second month after start of delivery order

Technical Documents Final documentation is due NLT End of Task Order (EOTO) and will incorporate changes resulting from Government review of the documents. Draft documentation is due on an “as required” basis.

Data Files Final versions of all analysis data files/datasets are due NLT EOTO.

Source Code Final documentation of all source code is due

NLT EOTO.

6.0 PLACE OF PERFORMANCE.

The place of performance will be at the contractor’s facilities or at OPA, Seaside, facilities, as required.

7.0 PERIOD OF PERFORMANCE.

The period of performance will be 12 months from award.

8.0 SECURITY CLEARANCE.

No classified work will be done under this delivery order; however, all forms and materials generated under this effort shall be treated as “Controlled Items-Test Material” and will be safeguarded and controlled under the Standard Operating Procedures of the Office of People Analytics (OPA), Defense Personnel Assessment Center. All individuals hired under this contract are required to sign a non-disclosure agreement that prevents disclosure of any details of the items that are analyzed and/or the forms that are developed as part of this effort to any outside agency other than the

Assisted Acquisition Services (AAS) Region 9

Contractor or the OPA. Copies of the non-disclosure agreements are to be provided to

OPA.

10.0 TRAVEL

Local or long-distance travel may be required to various locations, as directed by the Government, in order to perform the tasks delineated in this PWS. All non-local travel must be pre-approved by the government. Contractors shall not start any travel without prior approval by the government. All cost reimbursable travel shall be in accordance with FAR 31.205-46 and applicable travel regulations (Joint, Federal or Standardized).

Contractor payment claims shall include applicable documentation to support actual costs incurred (e.g. airfare and hotel/lodging receipts) as well as any receipts valued at or above $75.00. Failure to provide appropriate documentation may result in loss of reimbursement of travel expenses. The contractor shall notify the COR and obtain written authorization prior to travel. Anticipated travel is as follows:

10.1 Four trips for two people to OPA, Seaside, or locations of Manpower Accession

Policy Working Group and Defense Advisory Committee meetings are anticipated. The purpose of the trips are to coordinate analysis plans and review results or present work to oversight groups.

11.0 OTHER DIRECT COSTS (ODCs)

If applicable, the Contractor shall include Other Direct Costs (for any necessary software, etc.) in their proposal to cover any costs associated with ODCs. After award, all ODCs shall receive government COR approval prior to funds being expended. The COR must approve all requests for final ODCs prior to the contractor incurring costs. The actual pricing of the required ODCs provided by the Contractor will be validated by the COR. The Contractor shall attach back up documentation/receipts to the invoice.

Invoices for materials will not be paid until proper verification of backup documentation including purchase approval, competition documentation, and receipts.