ID09180038_D.1_ATTACHMENT_5_SAMPLE_TASK_ORDER__1.pdf

ID09180038 – Sample Task Order 1

D.1 ATTACHMENT 5_SAMPLE TASK ORDER 1 -

TECHNICAL SUPPORT FOR PSYCHOMETRIC SUPPORT

FOR THE ASVAB.

U.S. General Services Administration

Federal Acquisition Services Contracting Division

Contract Number _______TBD

ITSS Number_______________

Delivery Order Number: ______________

1. Title. Psychometric Support for the ASVAB.

2. Background. The Armed Services Vocational Aptitude Battery (ASVAB) is a multiple aptitude test battery used by all military Services to select and classify military applicants. The ASVAB is currently administered as both a paper-and-pencil (P&P) test and a computer adaptive test (CAT), although eventual elimination of P&P testing is a goal of the program. Hence, the development of new test forms/item pools is currently targeted toward CAT-ASVAB only. [Note that under ASVAB terminology, both P&P test forms and CAT item pools are referred to as “forms.”]

Elimination of P&P-ASVAB is likely to result in increased scrutiny of CAT-ASVAB, which could result in increased vulnerability of CAT-ASVAB to item/form compromise. Due to these concerns, OPA is expediting the rate of new form development, so that operational CAT-ASVAB forms can be replaced on a more frequent basis than is currently done. Because some ASVAB subtests are viewed as more vulnerable than others to compromise, the form development and replacement schedule will vary across the ASVAB subtests. The projected yearly form development rates are given in the table below for each ASVAB subtest.

Subtest

Number of New CAT-ASVAB Forms per Year

GS 4

AR 4

WK 8

PC 4

MK 4

EI 2

AI 2

SI 2

MC 2

AO 2

3. Objective. The objective of this effort is to develop CAT-ASVAB forms for eventual use in the

Enlistment Testing Program and the Career Exploration Program. The steps to developing new CAT-ASVAB forms include cleaning and scoring CAT-ASVAB data, conducting item analyses and screening items, assembling item pools, and equating test scores across new and existing CAT- ASVAB item pools.

4. Specific Tasks:

4.1. Data Management. The contractor shall process and clean CAT-ASVAB data for purposes such as eliminating duplicate records, eliminating corrupt records, eliminating records with invalid responses, and retaining one random record for repeat testers. The contractor shall score the item responses and format the CAT-ASVAB data for use in activities such as item calibrations, parameter rescalings, and DIF analyses. The contractor shall also work with OPA staff to verify/finalize item keys prior to scoring the response data and conducting item calibrations.

4.2. Item Calibrations and Parameter Rescalings. The contractor shall conduct item calibrations using calibration software designated by OPA. The contractor shall rescale the resulting item parameters to be on the operational CAT-ASVAB scale, using an approach designated by OPA.

4.3. Item Analyses and Screenings. The contractor shall compute item analyses specified by

OPA, including classical item statistics, distractor analyses, DIF analyses, IRT item information functions, and IRT item fit statistics. The contractor shall make keep/drop decisions for all items based on classical and IRT-based item analyses. The contractor shall also work with OPA to identify items flagged for non-statistical reasons that will be excluded from the analyses described in paragraph 4.5, and to update OPA’s item history files to reflect keep/drop decisions.

4.4. Create Enemy Groups. The contractor shall identify enemies and create enemy groupings for each ASVAB subtest prior to building final new forms/pools. The contractor shall work with DMDC to develop extensive enemy groupings for the Mathematics Knowledge and Mechanical Comprehension subtests, based on content evaluations/comparisons. Enemy groupings for the other ASVAB subtests will be based on documentation provided by OPA.

4.5. Build Forms/Pools. The contractor shall build preliminary and final parallel test forms by subtest from a set of available items identified by OPA. Final pools will take into account item enemies as identified in paragraph 4.4. The contractor shall evaluate the item pools by conducting simulations and computing score information functions, as well as comparing the precision to appropriate P&P-ASVAB and CAT-ASVAB forms. Final evaluations of item pools will also entail a key check, plus evaluations of content coverage and within form item enemies.

4.6. Conduct Preliminary Equating. The contractor shall conduct a preliminary theoretical equating of the new adaptive test forms to the appropriate operational and reference CAT- ASVAB test forms. The preliminary equating will be used during an Initial Operational Test and Evaluation of the test forms, which will provide the data for a final equating.

4.7. Conduct Final Equating. The contractor shall conduct a final empirical equating of the new adaptive test forms using data collected in a random groups equating design. This equating will be used to score the exams of applicants tested subsequent to the final equating implementation.

4.8. Procedure Refinement. Refine statistical and analytic form development procedures.

These refined procedures will provide more accurate and efficient methods for data management, item calibration and scaling, item screening, enemy group construction, pool assembly, impact of keeping/dropping misfitting items, and equating.

5. Deliverables and Reporting Requirements.

5.1. Monthly Status Reports. The contractor shall document the status of the effort each month.

5.2. Technical Documents. The contractor shall provide written documentation of steps taken and outcomes for each task, such as you would find in a technical manual or technical bulletin.

5.3. Data Files. The contractor shall provide copies of all data files/datasets that are created and/or used in the analyses.

5.4. Source Code. The contractor shall provide written documentation of all computer programs, to include flow charts of all main and other routines. System documentation shall include electronic versions of computer programs. Computer source code shall be delivered in electronic format.

6. Schedule of Deliverables.

Deliverable Due Date Monthly Progress Report Monthly, starting with the second month after start of delivery order Technical Documents Final documentation is due NLT End of Task

Order (EOTO) and will incorporate changes resulting from Government review of the documents. Draft documentation is due on an “as required” basis.

Data Files Final versions of all analysis data files/datasets are due NLT EOTO.

Source Code Final documentation of all source code is due

NLT EOTO.

7. Place of Performance. The place of performance will be at the contractor’s facilities or at OPA, Seaside, facilities, as required.

8. Travel: Local or long-distance travel may be required to various locations, as directed by the Government, in order to perform the tasks delineated in this PBSOW. All non-local travel must be pre-approved by the Government. Contractors shall not start any travel without prior approval by the Government. All cost reimbursable travel shall be in accordance with FAR 31.205-46 and applicable travel regulations (Joint, Federal, or Standardized). Contractor payment claims shall include applicable documentation to support actual costs incurred (e.g.

airfare and hotel/lodging receipts). If travel costs are expected to exceed this amount, the contractor shall notify the COR and obtain written authorization (by the Contracting Officer) prior to travel. Contractor costs for Government authorized travel are included in this contract.

8.1. Travel: Four trips for two people to OPA, Seaside, are anticipated. The purpose of the trips is to coordinate analysis plans and review results.

9. Period of Performance. The period of performance will be 12 months.

10. Security Clearance. No classified work will be done under this delivery order; however, all forms and materials generated under this effort shall be treated as “Controlled Items-Test Material” and will be safeguarded and controlled under the Standard Operating Procedures of the Office of People Analytics (OPA), Defense Personnel Assessment Center. All individuals hired under this contract are required to sign a non-disclosure agreement that prevents disclosure of any details of the items that are analyzed and/or the forms that are developed as part of this effort to any outside agency other than the Contractor or the OPA. Copies of the non-disclosure agreements are to be provided to OPA.