RRL_FY24_CC_SOW_App_C_I_Aligned_28Feb23.pdf

28 February 2023 APPENDIX C Aligned with APPENDIX I: RRL FY24 Content Conversion Ratings

Appendix C (Terms and Definitions) Aligned with Appendix I (Media Characteristics)

SOW Appendix C – Terms and Definitions SOW Appendix I – Media Characteristics

Level 1 IMI (Typically instructor facilitated) Instructor Interactions: Low to moderate; performs basic interactions with the delivery system (e.g. paging through content).

Student Interactions: Passive to limited participation;

may perform basic interactions with the delivery system as directed.

Audiovisual Media: Text; photos; video/audio (if customer-supplied or if minimal production and postproduction is required, e.g. can be recorded using simple device and little or no editing necessary); 2D/3D Graphics (not complex): repurposed or customer-supplied complex graphics (where no editing or revision is necessary): interactive GUI (menus, submenus); minimal hyperlinks/hotspots; customer-supplied animations; simple animations of parts/equipment/process flows; check-on-learning animations.

Menu/Path Includes: Navigation is primarily linear with occasional simple menus/ submenus to one or two paths and then return (e.g. moving page-to-page by clicking on the “Next” button or on objects that advance the presentation in a linear path (i.e. page-turner)); however, may be designed for the user to be able to respond to instructional cues (i.e. objects on the screen such as point-and-click objects, rollover objects, and drag-and-drop objects). Using hotspots or blue robs to advance the screen is the same as clicking "Next" to advance (still essentially a linear progression). Designed as an Information-Only or an Information-Plus-Demonstration strategy.

Learning Activities: (Include but are not limited to) Practice activities with feedback are limited to recall of information presented or are separately directed as lab activities.

Comprehension Checks: (Include but are not limited to) Multiple choice, matching, etc. with immediate instructor feedback or system-generated feedback as appropriate.

Media Characteristic Definition

IMI 1

Text (Self-explanatory)

Audio

Can include voiceover narration and sounds corresponding with visual presentations, animations, audio clip recordings, mechanical sounds (e.g., beeps, buzzes, clicks) and replications of natural environmental sounds.

2D Graphic

Highly detailed image. High resolution (2D graphics). High level of detail and focus in visual representation/replication. Can be depicted in a 2D or 3D graphic.

3D Graphic

Exploded view and/or view all sides of object (e.g. system, component).

Provide a visual depiction in an X-Y-Z scale (width, height, depth) to depict components within a larger system (“exploded” view) or to provide a rotational view of all sides of a component or system. Unlike object modeling, a 3D graphic is static and does not allow user interaction with the image.

Motion Graphic/Video (Self-explanatory)

Level 2 IMI (Typically self-directed, but can be instructor facilitated) NOTE: Given current technology and considering the most basic computer use, almost all self-paced IMI falls into Level 2, which relies heavily on Level 1 elements e.g.

mostly linear presentation and simple menus to one or two paths) but with audiovisual elements developed at a more complex level. The exception would be Level 2 IMI with embedded Level 3 learning activities (e.g. uses simulations to perform procedural skills, or application of principles such as tactics. In such cases, the Level 3 learning activity elevates to an IMI 3. Level 2 often provides the knowledge component as prerequisite to a lab activity).

Media

IMI 2

Item Selection Interaction with courseware (e.g.

menus, hotspots, modeled objects, etc.,) requires inputs/selections to occur one at a time.

Procedural Response

Input actions performed in sequence or step-by-step.

Object Modeling

1. Static Models – 3D models that look the same as if in a live setting, with realistic proportions, color, textural appearance, etc. Interaction with static models allows functionality such as rotation of the model to see all angles and points of view, selecting

Student (or Instructor) Interactions: Moderate to complex participation; performs moderate to complex interactions with the delivery system.

Audiovisual Media: Video/audio (if minimal production and postproduction is required, e.g. can be recorded using simple device and little editing necessary); 3D Graphics;

interactive GUI (menus, submenus); hyperlinks/hotspots;

animations of parts/equipment/process flows; check-on-learning animations. Active object modeling that includes all the features of static models and that includes motion to react and function as they would in a live setting, with the correct speed, acceleration, trajectory, etc. Interaction with active models at the IMI 2 level allows manipulating the model and/or interacting with it in a demonstration mode, e.g. to identify components and their locations, identify controls and indicators, etc. and to see their basic functions in normal operation mode.

Menu/Path Includes: Navigation is based on instructional cues (i.e. student prompts, presentation of stimulus). The learner makes simple responses to instructional cues and interacts with objects on the screen such as point-and-click objects, rollover objects, and drag-and-drop objects (i.e. simple item selection, procedural response).

Interaction offers feedback and remediation. The learner has more control over navigation with two or three menu/path capability. Designed as an Information-Only or an Information-Plus-Demonstration strategy.

Learning Activities: (Include but are not limited to) Procedural skills demonstrated by the courseware and procedural skills via previously unencountered scenarios, both with immediate feedback.

Comprehension Checks: (Include but are not limited to) Multiple choice, matching, etc. with immediate system-generated feedback, and previously unencountered procedural skills scenarios.

hotspots to see a close-up view of a component or a popup text box, etc.

2. Active Models – 3D models that include all the features of static models and that include motion to react and function as they would in a live setting, with the correct speed, acceleration, trajectory, etc. Interaction with active models at the IMI 2 level allows manipulating the model and/or interacting with it in a demonstration mode, e.g. to identify components and their locations, identify controls and indicators, etc. and to see their basic functions in normal operation mode.

Level 3 IMI (Typically self-directed) Student Interactions: Complex participation; performs complex interactions with the delivery system.

Audiovisual Media: Video/audio; 2D/3D Graphics (complex); interactive GUI; coded activities (e.g.

embedded as an interactive event within a shareable content object (SCO) – such as guided practice, emulation, games, simulation of parts of systems, animations, and Check on Learning. Can also be a standalone executable rather than embedded within a SCO. Also, nonlinear instructional activities which use emulation/stimulation/ actual system software (e.g. MQ-4C Triton actual system software used with training scenarios contained in pdf files).

Menu/Path Includes: This level of interactivity most often applies to specific embedded learning activities within an otherwise Level 2 lesson (e.g. a learning activity developed in gaming engine software and embedded as an executable within a linear lesson) to perform procedural skills, or application of principles such as tactics. Involves simulated activities such as a how-to guide for learning software; simulated activities depicting diagnostic procedures; simulated operational procedures;

Media Characteristic Definition

IMI 3 Object Modeling

1. Static Models – same as IMI 2.

2. Active Models – same as IMI 2, but interaction with active models at the IMI 3 level allows manipulating the model and/or interacting with it in a performance mode, i.e. to perform an operation or maintenance task, etc.

• The model is embedded as an interactive event within the navigation path. The model’s navigation controls are separate from the overall GUI’s navigation controls.

• The model is developed to teach, practice, or assess a single task or procedure.

• The event is often approached as guided practice with only one possible correct navigation path and simulated activities for maintenance, troubleshooting and repair. The learner controls the learning experience by responding to instructional cues (i.e. presentation of stimulus) which may involve multi-path navigation. The learner is encouraged to branch (test out or otherwise skip content already mastered), make decisions, and alter paths, and receives constructive feedback. The learner uses varied techniques in response to instructional cues involving complex concepts, procedures, and evaluation.

A lesson may present complex Operation and Maintenance procedure scenarios. Designed as an Information plus Demonstration plus Application strategy, or as a Whole Task-Centered with Demonstration Application strategy.

Learning Activities: (Include but are not limited to) Practice with immediate feedback are mostly application of procedural skills, with ample opportunities to practice, but not with free-play.

Comprehension Checks: (Include but are not limited to) Testing with tailored remediation; and adaptive branching based on pretest performance. There should be few, if any, recall of information questions in a Level 3 learning activity.

See IMI 3 Scenario/Simulation Types below for additional information.

and immediate, or three-attempts, corrective feedback. There is no requirement that limits the event to a guided practice – it can be other than guided practice too.

• There is no requirement that limits the assessment activity to repeat the guided practice. Ideally, the assessment will be a different scenario rather than a repeat of the guided practice.

Complex Controls (Interactive Objects)

Student interaction with courseware, object models, and/or simulated controls that allows actual or simulated system inputs with feedback or system responses relative to the scenario. E.g.

interacting with a modeled object or simulated controls to perform a task, interacting with simulated tools and their controls in addition to the primary modeled object, and interacting with multiple submenus or icons on a given screen that each provide additional information to the scenario, etc. Such interactions should include more than merely selecting or manipulating objects in a linear sequence, especially when prompted by direction text and there is only one possible active selection area at a time.

Complex Scenarios

Trainee inputs impact the scenario and cause the scenario to change, resulting in multiple ways an LO can be executed. E.g. complex scenarios allow a trainee to perform multiple, simultaneous actions (inputs), or to respond to a variety of variables or conditions, with system-generated feedback (responses) dependent upon the combined inputs. Complex scenarios are not restricted to a linear sequence, such as a lock-step guided practice activity with a stimulus-response-reward approach.

Performance based complex scenarios should not rely on multiple choice questions to advance the scenario.

Multi-path Navigation

The User Experience Design (UX) and User Interface Design (UI) allow navigation that includes access to all parts of the SCO via the use of menus, hyperlinks, pop-ups, multiple navigation panes, etc. Practice activities may be guided or unguided, as appropriate, for the scenario(s) and LO. Performance tests should not repeat the practice activity unless there is only one correct way to perform the LO and no alternate scenario(s) would be appropriate.

Navigation down an incorrect path shall be corrected within two steps.

Visual Motion Awareness

Sensation of moving through an environment relative to other objects.

Complex Media (i.e. Virtual Simulation or VSIM) NOTE: Augmented Reality (AR), Mixed Reality (MR), Virtual Reality (VR), and Immersive Virtual Environment (IVE) are complex media that differ from the requirements and characteristics of VSIM. They each have their own capabilities and functionality that is distinct from VSIM.

The following applies to VSIM.

NOTE: Any design for training with this type of complexity will follow a blended Software Engineering (SE) and Instructional Systems Design (ISD) process, and will therefore require a linked combination of specific SE and ISD CDRLs.

A computer-based representation of the function and operation of one or more systems or subsystems in context of an operational environment. VSIM uses mathematically accurate system models that replicate physical processes and duplicate system response (indicators) in real time based on physics based modeling of student inputs (controls).

Interaction with system simulation is generally conducted through realistic representations of the actual system control panels or interfaces. Simulation is a model-based environment that describes the correct state of any element in the environment through the definition of relationships and attributes.

A key distinction between VSIM and IMI 3 is the capability of the instructor to control setup and operation of the simulation via an instructor operator station (IOS) or an instructor workstation (IWS). For example, the capability to Set Training Mode (allows instructor to demonstrate system theory/normal operations) or Set Maintenance Training Mode (allows instructor to demonstrate any combination of multiple faults, and allows student to practice troubleshooting and maintenance procedures on any combination of fault conditions, as set by the instructor using the IOS or IWS).

Another example, the IOS or IWS allows the instructor to set different initial conditions in the simulation, e.g. all tanks empty, all tanks half-full, all tanks full, or all tanks just below pilot valve cut-off.

VSIM Type I

• SCORM compliant and will run from the Navy LMS across the internet, and via TRANET

• Generally, runs on a standard desktop or laptop PC with keyboard, monitor (may be dual monitors), and mouse

• Fidelity ranging from A to B

VSIM Type II

• Runs on a closed (standalone) system that is not

SCORM compliant and will use another form of student tracking than the Navy LMS (e.g.

Media Characteristic Definition

Complex Media

Object Modeling

1. Active Models – same as IMI 3 (especially with Complex Controls, Complex Scenarios, and Multi-path Navigation), but adds one or more of the Complex Media characteristics, which increases the fidelity of interactions with the model.

The model is developed to teach, practice, and assess more than one task or procedure. Interaction with active models at the Complex Media level allows manipulating the model and/or interacting with it in a performance mode, i.e. to perform operation and maintenance of multiple tasks in multiple modes.

• The model is the primary focal point of instruction, practice, or assessment. It is not embedded as an interactive event within an otherwise linear navigation path.

• The model is developed to teach (demonstrate), practice, or assess multiple tasks or procedures.

• The model’s primary navigation controls are part of the model (i.e. use the simulated system controls to interact with the system), with multiple active hotspots or touch points at the same time.

Realistic Physics (pertaining to the object being modeled)

Modeled objects behave realistically in response to gravity, friction, or force.

Objects convey accurate and realistic, natural, physical laws (not gaming properties), such as feeling the turn of pliers or the resistance of a piece of copper wire being twisted, or seeing the gravity effect on a flange safety shield being unwrapped and wrapped around a pipe. Modeled objects respond to gravity, Multipurpose Reconfigurable Training System (MRTS 3D) or similar)

• Runs on a PC plus multiple monitors (usually touch screens)

• May include the use of haptic or hardware controllers, the use of multi configurations of displays or user-interface devices, and may require additional servers to control multiple instances or provide image generation for 3D environments or entity visuals.

• Fidelity ranging from B to C

VSIM Fidelity Levels:

A (Low) – Use for familiarization, recall, and application.

Contains still graphics, cutaways, and vector-based animations. Models system functionality, but not system states. Provides only fault-free system simulation (i.e.

models one or more normal operation conditions). The simulation is based on empirical inference.

B (Medium) – Use for application, analysis, synthesis, problem solving, and transfer. Contains 2D/3D animations.

Offers both fault and fault-free simulation of system components (i.e. models more than one normal operation conditions plus more than one fault conditions and/or degraded modes of operation). Has correct modeling of system static states within actual system tolerances.

Simulation depicts some dynamic system states.

C (High) – Use for problem-solving, transfer, and as a job performance aid. Contains complex equipment modeling.

Has a mathematically correct component-based model.

Accurately simulates dynamic states, faults and fault effects of individual components. Replicated system static and dynamic response is within tolerances prescribed for actual equipment.

Student Interactions: Complex participation; performs complex interactions with the delivery system.

Audiovisual Media: Video/audio; 2D/3D Graphics and animations (complex).Activities (e.g. demonstration, practice, and assessments) are the primary focal point of the simulation, not embedded in an otherwise linear navigation path.

Menu/Path Includes: This level of interactivity applies to specific performance activities such as performing procedural skills or application of principles such as tactics. Often involves simulated activities depicting operational procedures, diagnostic procedures, and troubleshooting. The learner controls the learning experience by responding to instructional cues (i.e.

presentation of stimulus) which may involve open-ended navigation. The learner is encouraged to make decisions, and alter paths, and receives feedback. The learner uses varied techniques in response to instructional cues involving complex concepts, procedures, and evaluation.

A learning event may present complex Operation and Maintenance procedure scenarios. Designed as a Demonstration plus Application strategy, or as a Whole Task-Centered with Demonstration Application strategy.

Learning Activities: (Include but are not limited to) Demonstration and application of procedural skills (with ample opportunities to practice) including performance of collisions and friction in the same way they would in the real world.

Out-of-tolerance Feedback

The required actions to execute the LO are constrained by limiting factors, impacting the way the LO is performed. These limiting factors must be considered and accounted for in performance of the LO (e.g., actions must be taken within a specific time frame, at precise times or locations, performed with a specific amount of pressure or at a certain pace or rhythm, occur within other defined thresholds, etc.).

NOTE: Out-of-Tolerance Feedback is different from Fault Insertion and from using a fault condition as a scenario (e.g. a learning activity based on a single fault condition, or one fault condition at a time over a serial learning activity);

and from immediate, or three-attempts, corrective feedback.

Multiple Fault Insertion

Capability of the training system to allow the instructor to insert any combination of multiple faults or discrepancies that each cause divergence from normal functioning of the system or scenario being used, requiring corrective action from the trainee to return the system to normal functions.

NOTE: Fault Insertion is different from Out-of-Tolerance Feedback and from a learning event that only deals with a single fault condition or a menu allowing the student to work through a mandatory set of fault conditions (fault scenarios) – where the overall activity is still forced in order to record the event as completed.

3D Immersive Environment

3D Immersive field-of-view.

Provides an extended horizontal and vertical field-of-view (beyond the 180o field seen on standard flat-screen projections). This extended field of view is different from simply rotating the object, or viewing the object inside a normal operations, fault conditions (i.e. troubleshooting and repair), and degraded modes of operation. Free-play is often allowed, though perhaps with some restrictions.

Comprehension Checks: (Include but are not limited to) performance assessments.

simulated environment. It is an extended view of the space (i.e. room) and capability to move within that space on an X, Y, and Z axis, e.g. simulates the student moving through the space or having peripheral vision within the space beyond the normal field of view.

Realistic Terrain

Environmental surroundings impact responses or reactions.

Replicates specific terrain because the type of terrain impacts how the LO must be executed.

Physical Environmental Effects (pertaining to the space or natural environment within which the modeled object is placed)

Environmental objects behave realistically in response to gravity, friction, or force and thereby impact the modeled object (e.g., sea state and weather affects the small boat, etc.). Convey accurate and realistic, natural, physical laws (not gaming properties).

Environmental objects respond to gravity, collisions and friction in the same way they would in the real world.

Reporting Capabilities

The ability to view, review or document responses during practice or performance of an LO. Reporting capabilities are more than a mere checklist of process steps competed.

Scenario Playback

The capability to record student performance to allow for replay and review of that performance after the LO has been executed.

Complex Media (with Hardware) Media

Complex Media w/ Hardware

Tactile, Feel, and Haptics

Tactile, feel and/or haptic responses convey accurate and realistic representation of the sensations produced by physical contact of an object, instrument or natural element in the environment (e.g., surface contact, pressure, feedback, etc.).

Actual Controls

The actual or operational equipment/controls must be available as part of the training package or a high-fidelity simulation of the control characteristics needs to be developed for training.

Complex Media (Operational Equipment;

Modified Operational Equipment) Media

Actual

(Operational) Equipment

Training Equipment (Self-explanatory)

IMI 3 Scenario/Simulation Types

For each learning objective recommended for a level of complexity at IMI 3, the contractor shall design and develop the SCO using one or more of the following scenario types, as appropriate, to teach (present instruction), practice (with relevant feedback), and test the learning objective:

1. Analysis / Apply Rules / Problem Solving – dealing with unknown or previously unencountered situations; responding/interpreting; demonstrating mental preparedness to encode operational cues as indicators of normal, abnormal, and emergency conditions associated with the performance of an operational procedure.

2. Procedural Knowledge / Follow Procedures – Following step-by-step instructions.

3. Demonstrate / Perform / Complete a Task – Performing an operator task, maintenance task (e.g.

service, maintain, repair, remove/install), or troubleshooting task on a system or component of a system.

The SCO models normal operating state and pre-set fault condition(s) (not the same as complex media, which also models fault conditions that are not pre-sets).

4. Apply a Theory / Demonstrate Conceptual Knowledge and Act on It – Categorizing knowledge by attributes.

5. Cause and Effect (Stimulus/Response) Demonstration – for example, manipulating an object to see the reaction (e.g. push the lever down and the weight goes up; normal operation of a system and/or its components).

Each IMI 3 scenario type (i.e. SCO) shall exhibit these media characteristics at a minimum:

• Item Selection

• Procedural Response

• Object Modeling – the SCO presents 2D and/or 3D object model(s) which the trainee can manipulate to perform the work or learn the concept.

o Model is embedded as an interactive event within the navigation path. Model’s navigation controls are separate from the overall GUI’s navigation controls.

o Model is developed to teach, practice, or assess a single task or procedure within a single interactive event.

o Event is often approached as guided practice with only one possible correct navigation path and immediate, or three-attempts corrective feedback. NOTE: There is no requirement that limits the IMI 3 event to a guided practice – it can be other than guided practice too.

o There is no requirement that limits the assessment activity to repeat the guided practice.

Ideally, the assessment will be a different scenario rather than a repeat of the guided practice.

• Complex Controls (via GUI and navigation structure) - the SCO allows interaction with the courseware via various input methods with various feedback or responses from the SCO for the different inputs.

• Complex Controls (Interactive Objects) - the SCO provides or simulates complex controls that the trainee needs to interact with in order to simulate performing the work, e.g. one or more control elements that allow actual or simulated system inputs with feedback or system responses relative to the scenario.

• Complex Scenarios - the SCO allows trainee actions to impact the scenario, where the scenario can change based on trainee inputs, resulting in multiple ways the LO can be executed. The SCO provides scenario(s) needed to practice the work, e.g. the SCO requires the trainee to perform multiple, simultaneous actions or to respond to a variety of variables or conditions. NOTE: text-based multiple choice or decision tree type scenarios are not considered complex scenarios since they do not simulate actions.

• Multi-path Navigation – the User Experience Design (UX) and User Interface Design (UI) of the SCO allows navigation that includes access to all parts of the SCO via the use of menus, hyperlinks, pop-ups, multiple navigation panes, etc. Practice activities may be guided or unguided, as appropriate, for the scenario(s) and LO. Performance tests should not repeat the practice activity unless there is only one correct way to perform the LO and no alternate scenario(s) would be appropriate. Navigation down an incorrect path shall be corrected within two steps.