Q_As RFI OUSD(C) DFMC DL.pdf

Attachment 1

Question/Comment

Number

1 Will the contract be Firm Fixed Price?

OUSD(C) Response: Yes, the contract will be Firm Fixed Price

2 There are conflicting statements as to whether all or some of the courses must be delivered at Level of Interaction (LOI) III.

• The RFI page 3, Task Area #2, states all courses.

• However, the Performance Work Statement (PWS) in

Section 2.3 states “all or some of the courses.”

• The PWS in Section 4.1.1 states all courses.

• But then, the PWS Section 4.1.5.2 states “all or some of the existing courses will be upgraded to LOI III.”

• The PWS, Section 4.1.5.3, states that “RC calculations will be used for courses that will not be transitioned from legacy

LOI II to LOI III.” This supports the “some or all” guidance.

Please clarify whether all or some of the courses must be delivered at Level of Interaction (LOI) III.

OUSD(C) Response:

• RFI: The RFI, in the objective, states, “This initiative will result in 1) Increasing the LOI of all or some of the courses from LOI II to LOI III…”. This is stated correctly.

• RFI Critical Task Areas: No. 2, PWS 4.1, states, “Deliver all courses at LOI III as defined in Para 4.1.5.2. PWS Para 4.1.5.2 states, “All or some of the courses will be upgraded to

LOI III. This is stated correctly.

• PWS Para 2.3 states all or some as it should.

• PWS Para 4.1.1. Revised PWS to state “all or some of courses.”

• PWS Para 4.1.5.2. States all or some of the courses.

• The customer is still determining if ALL courses will be refreshed from LOI II to LOI III.

• RC calculations is correct as is.

3 The PWS, Section 4.1.5.2, states “All existing FWM courses are

LOL II.” This statement is misleading. While it is accurate that courses under the current contract are Level II, this statement applies to the LOI II definition under that current contract. This

PWS clearly adds requirements to all LOIs and in that context it is not certain that all current courses are Level II. The added requirements are a major concern, as will be discussed in the next

Question/Comment.

• OUSD(C) has revised the LOI table. Refer to PWS Para 4.1.5.2.

4 Why does the PWS add requirements to the LOI, as shown on pages 7 through 9 of the PWS? It appears that the additions stem from the Army DL Program/TRADOC Pamphlet 350-70-12.

Although this Army Pamphlet is recent, it clearly does not apply to non-Army organizations. Below is an extract from the cover Page of the Pamphlet:

Applicability. This pamphlet applies to U.S. Army Training and Doctrine Command activities and One Army School System training battalions responsible for managing, developing, and implementing learning products. It also applies to non-U.S. Army Training and Doctrine Command agencies and organizations possessing memoranda of understanding, memoranda of agreement, and contracts for developing learning products for U.S. Army Training and Doctrine Command and One Army School System agencies and organizations.

These Army requirements surpass those of the current DoD guidance

MIL-HDBK-29612 “Development of Interactive Multimedia

Instruction (IMI) Part 3 of 5 Parts)”, which was validated at the DoD level in December 2020. ASSIST-QuickSearch Document Details (dla.mil)

These Army requirements are also out of step with industry standards for LOI levels. See:

Association for Talent Development, “How Long Does It Take to Develop Training? New Question, New Answers.” (Table 1 shows the different levels of E-Learning interaction) (https://www.td.org/insights/how-long-does-it-take-to-develop-training-new-question-new-answers)

Articulate Global/E-Learning Heroes, “Get to Know The 4 Levels of E- Learning,” (https://community.articulate.com/articles/get-to-know-the-4-levels-of-e-learning https://www.shiftelearning.com/blog/bid/190140/Levels-of-Interactivity-in-eLearning-

Which-one-do-you-need https://elearningindustry.com/levels-of-interactivity-elearning-modules https://elearningindustry.com/levels-of-interactivity-in-elearning-advantages-4 https://blog.elblearning.com/the-four-levels-of-elearning-interactivity https://enyotalearning.com/blog/elearning-interactivity-levels/ https://www.talentlms.com/blog/elearning-interactivity/ https://www.windwalker.com/news/2022/4/5/levels-of-interactivity

4 levels of interactivity in elearning instructional design (learningeverest.com) https://quicksearch.dla.mil/qsDocDetails.aspx?ident_number=207795 https://www.td.org/insights/how-long-does-it-take-to-develop-training-new-question-new-answers https://www.td.org/insights/how-long-does-it-take-to-develop-training-new-question-new-answers https://community.articulate.com/articles/get-to-know-the-4-levels-of-e-learning https://community.articulate.com/articles/get-to-know-the-4-levels-of-e-learning https://www.shiftelearning.com/blog/bid/190140/Levels-of-Interactivity-in-eLearning-Which-one-do-you-need https://www.shiftelearning.com/blog/bid/190140/Levels-of-Interactivity-in-eLearning-Which-one-do-you-need https://elearningindustry.com/levels-of-interactivity-elearning-modules https://elearningindustry.com/levels-of-interactivity-in-elearning-advantages-4 https://blog.elblearning.com/the-four-levels-of-elearning-interactivity https://enyotalearning.com/blog/elearning-interactivity-levels/ https://www.talentlms.com/blog/elearning-interactivity/ https://www.windwalker.com/news/2022/4/5/levels-of-interactivity https://www.learningeverest.com/interactivity-in-elearning-instructional-design/

This is a critical issue for the PWS since the added Army requirements are substantially more complex and therefore more difficult and costlier to develop. The added requirements for LOI III repeatedly refer to complex operational procedures, operational setting, environment simulating decision-making events. The Army requirements have a definite flavor of tactical training of units in an operational environment, such as a platoon of tanks in a simulated environment, or an artillery crew firing in an operational setting.

These Army LOI III are not suited for financial management training, and if forced fit, such training will be exorbitantly expensive to develop, without the expected return on investment. It is notable that the Army document states that “Many COEs (Centers of

Excellence) and schools do have the capability to develop Levels 0 and 1.” But, for higher level LOI courses, it must compete for funding to engage a centralized contractor. This training is often complex, involving infrastructure simulators, integrated communications, and real-time scenarios. As another unworkable example, please see the second-to-last new requirement “The learner operates in a game-like environment that may involve other players and/or computer-generated team players.” The current environment for equipment needed by learners is simply a computer with a valid

CAC. To meet this requirement, the system hosting the learning would need to be able to support the robust capabilities of seeing other learners that are online and available as well as providing a method for the learners to interact with each other.

There is also another major problem with the LOI III requirements, specifically compliance with NASBA standards. The first added LOI

III requirement is “Exercise total control over the presentation. The learner operates in the environment without instructional cues.” This feature can result in wide variations for how long the training goes on, which will not fit with CPE calculations stipulated by NASBA. In effect, one learner may shortcut the branches, while another explores every branch. How do we credibly measure the number of CPEs advertised and granted?

We strongly recommend that the added LOI III requirements be removed from the PWS. OUSD(C) should use the LOI III guidance from the current DoD guidance MIL-HDBK-29612 “Development of

Interactive Multimedia Instruction (IMI) Part 3 of 5 Parts)”, which was validated at the DoD level in December 2020. The government will still receive enhanced courses but without exorbitant costs and very possibly infeasible solutions.

• OUSD(C) has revised the LOI table. Refer to PWS Para 4.1.5.2.

5 Question: Will OUSD(C) require the contractor to provide a

Refresh Complexity (RC) calculation and rationale?

PWS Section 4.1.5.3 states that “RC calculations will be used…”

However, the last sentence of this Section states “the contractor will use the OUSD(C) provided RC.” Why must the contractor calculate the RC, if OUSD(C) provides the RC? If the contract is to be Firm Fixed Price, then the OUSD(C) provided RC will be used in the fixed price submitted by the bidder. Any RC calculation by the contractor is superfluous and confusing.

Recommend that language be clarified to indicate that the contractor is not required to calculate the RC.

• The government will not require the contractor to provide an RC calculation.

• The government will require the contractor to provide a RC rationale in the form of the

Refresh Analysis Report/Deliverable 11. This requirement is detailed in PWS Para 4.5.2.

• Para 4.1.5.3: The vendor will use the RC provided by the government as stated in this paragraph.

6 Question: Since OUSD(C) provides the RC, why must the contractor provide the rationale for the RC?

PWS Section 4.5.2 states that the contractor is to perform the

Refresh Analysis and that “The Refresh Analysis Report

(Deliverable 11) shall provide the results of the Refresh Analysis, identify the Refresh Complexity, provide rationale for the Refresh

Complexity, and provide the price to perform the refresh using the formula from PWS Section 8.3.”

Recommend that the clause “provide rationale for the Refresh

Complexity” be deleted.

OUSD(C) Response:

• The government will not be removing the requirement for Deliverable 11/Refresh analysis report that explains the rationale for the refresh.

• PWS Para 4.5.2: The Refresh Analysis Report (Deliverable 11) shall provide the results of the Refresh Analysis, identify the Refresh Complexity (Per the government), provide rationale for the Refresh Complexity, and provide the price to perform the refresh using the formula from PWS Section 8.3.

7 PWS Sections 8.4 and 8.5 refer to the term “RC Units” but the term is undefined elsewhere in the PWS.

Recommend that RC Units be defined in these Sections as

“RC Units = Number of Course Hours x RC”

• Refer to revisions in PWS Para 8.4 and 8.5.