ID06180011_SF1449_Exhibit_05_Questions_and_Answers_08232018.pdf

SOLICITATION ID06180011

Exhibit 005

ID06180011 – Solicitation Vendor Questions and Government Answers – 08/23/2018 Project Name: DLIFLC Psychometric Analysis

RFP Question and Answers for GSA Solicitation ID06180011

Question

Solicitation Reference Vendor Question Government Answer

PWS C.5.5 Please clarify whether the facilitation of standard setting study panels is part of this scope of work, or whether the scope of the work is limited to analysis of existing panel recommendations.

Facilitation of standard setting study panels is NOT part of this scope of work. The scope of work is limited to analysis of existing panel recommendations.

2 PWS C.5.5 Regarding the psychometric analyses of the standard setting study panel recommendations, how many cut scores for each test are to be evaluated?

Generally, six cuts are evaluated for each test.

3 PWS C.5.4, C5.5 How long does the contractor have from the time the data is made available for each item pool until the analyses for that item pool must be completed?

The Government’s expectation is a turn-around (from contractor receipt of data to government acceptance of final report) of 60 – 90 days.

4 PWS C.5.4, C5.5 Is it anticipated that the analyses of each performance period that the schedule of analyses will be staggered for the different item pools or will the schedules be simultaneous?

For general staffing purposes, the schedule of analyses will be sufficiently staggered such that the number of psychometricians beyond that proposed for one analysis will not be required simultaneously. Refer to the revised PWS dated 08/23/2018. Technical Exhibit 01 has been added on page 14 which is a Base Year Projected Data Delivery Schedule (Government to Contractor) for the Number of Item Pools by Contract Task and Item Type.

5 PWS C.5.4 We currently interpret the RFP to indicate that the required analyses will be a subset (up to 14 MCT and 10 CR) of the item pools listed under 5.4.2, rather than all of the listed item pools. Is this understanding correct?

Yes. This is correct.

6 PWS C.5.4 Please clarify whether a single item pool should be interpreted as including both listening comprehension and reading comprehension for a single language, or whether listening comprehension and reading comprehension each count as a separate item pool.

Listening comprehension counts as a single item pool and reading comprehension counts as a separate item pool for each language. That is, Korean lower range listening comprehension and Korean lower range reading comprehension item pools, for example, are counted as two distinct pools for the purposes of this contract.

7 PWS General Are there limitations on which components of the work may be done by subcontractors?

DLIFLC places no limitations on which components of the work may be done by subcontractors.

8 PWS C.5.2 Other than the Project Manager and psychometricians, should any other staff be listed by name proposal, with their CVs provided?

This information is not required for the proposal. Note however that the successful “contractor shall submit Government provided confidentiality agreements for all personnel having contact with test material prior to working on the contract.” (C.6.5)

9 PWS C.5.2.1 Standard Setting Study Facilitators are listed as key personnel, yet the PWS has no mention of standard setting facilitation.

Must we have a member of our team identified as a Standard Setting Study Facilitator?

Refer to the revised PWS dated 08/23/2018. The reference to “Standard Setting Study Facilitators” at C.5.2.1 has been changed to “Psychometricians”.

10 SF1449

Solicitation Section L

Must we submit paper copies of the proposal?

No, only an electronic submittal of all stated proposal volumes is required.

11 PWS C.6.9 “Initial quality control requirements will be administered by the Program Manager.”

Is the ‘Program Manager’ the same as the ‘Project Manager’ defined as one of the key personnel?

Refer to the revised PWS dated 08/23/2018. The reference to “Program Manager” at Standard Setting Study Facilitators” at C.6.9 has been changed to “Project Manager”.

12 PWS C.10 “Contractor shall comply with Army Regulation 611-5 (2008 revision), paragraphs 3-1 and 3-2, as well as DLIFLC regulation 611-1 (dated July 1, 1997), paragraph 7 and 8.”

Will you please provide links to these regulations or share more detail?

Copies attached to the solicitation at Exhibits 06 & 07.

13 PWS C.5.4.4 How are the CRT items scored? Is there an automated scoring mechanism?

CRT items are scored dichotomously as “correct or incorrect.” Partial credit is not awarded. Scoring is not automated.

14 PWS C.5.4.4 (c) Does DLI have a preferred IRT model? 3PL for selected response, 2PL for constructed response.

15 PWS C.5.5.1 What type of psychometric analyses are you seeking for the standard setting panel recommendations?

Required analyses are stipulated in C.5.5.4. Analyses developed for DLIFLC (e.g., logistic smoothing and guessing adjustment heuristic, and a Bayesian model-based approach) will be shared with the contractor upon award.

16 PWS C.10.4. Can you explain how, where and how much time the OPSEC Training should take?

The OPSEC training is a web-based course available at the Government website address listed at C.10.4.

The course takes approximately 45 minutes.

17 SF1449 Exhibit 02 (Past Performance Questionnaire)

Is the requirement for one past performance questionnaire to be completed?

Please refer to Volume II Past Performance proposal instructions in Section L of the solicitation (starting on page 61). Paragraph 2.2.4 on page 62 states that “In addition to the Past performance narrative in Volume II, the offeror shall provide the separately attached Past Performance Questionnaire (PPQ) (Exhibit 02) to the points of contact for the contracts/task orders referenced in the narrative.”

It is the responsibility of each vendor to provide the PPQ to the contract points of contract identified in their Volume II narrative for completion and submittal directly to GSA.

18 SF1449 Exhibit 04 (Pricing Spreadsheet)

Travel cost of $30,000 is already preloaded in spreadsheet was that intentional and an estimate?

A pricing estimate of $30,000 has been filled by the Government for all travel CLINs (0005. 1005, and 2005). Refer to SF1449, Section 8, paragraph 8 on page 3 of the solicitation for more information.

19 PWS C.5.4 &

C.5.5

Are there typical sample sizes and bank sizes for each language proficiency test?

Item bank sizes vary from approximately 200 to 1,000 (400 average) items or greater per language and skill.

Examinee sample sizes vary from 200 to 30,000 (3,000 average) examinees or greater per language and skill.