M1 TEST PLAN AND TEST POINTS.pdf

Joint Terminal Control Training and Rehearsal System

(JTC TRS)

System Evaluation Test Plan

Version 1 14 February 2020

Prepared by:

AFLCMC/WNS

2300 D Street

Bldg 32 Wright-Patterson AFB OH 45433-7249

DISTRIBUTION STATEMENT D. Distribution authorized to the Department of Defense and U.S. DoD contractors only (Critical Technology) 14 February 2020. Other requests shall be referred to AFLCMC/WNS.

1. Test and Evaluation Strategy Overall: Given the government’s acquisition strategy to procure a non-developmental production-ready Joint Terminal Control Training and Rehearsal System (JTC TRS) compatible device, the government will verify the proposed solution meets the JTC TRS requirements. Systematic testing of an offeror’s JTC TRS compatible solution, which complements the technical evaluation of their proposal, will be accomplished in accordance with the source selection process identified in Sections L&M of the Request for Proposal (RFP).

A JTC TRS compatible solution is defined as a system which meets the JTC TRS FoS P-Spec, implements a Form, Fit, Function hardware baseline, and maintains the current software baseline. The Governments preferred solution should maintain the main software suite of MACE, VRSG, ASTi, and DIScover. Due to the nature of commercial software licensing, if vendors are unable to reach a business agreement, the Government will allow alternate software to be tested. Replacing MACE or VRSG will require testing in a dome environment to ensure the system integration and visual fidelity requirements are met. Replacing ASTi or DIScover will require testing in a Desktop environment in order to ensure system integration requirements are met.

Reference technical data provided in the Bidder’s Library for details of each device configuration.

1.1 Test Objectives

Through a combination of pass/fail criteria on critical and non-critical test points of the JTC TRS compatible system, testing, examination of critical peripheral equipment (i.e.

the Pilot Station Console (PSC), (Instructor/ Operator Station) (IOS), and Emulated Military Equipment (EME)), and execution of multiple operational training scenarios in the simulator, the government intends to verify that an offeror’s device satisfies requirements of the Performance Specification (P-Spec). Regardless of specific elements examined during the System Evaluation Test, the awarded offeror remains responsible to satisfy the P-Spec in its entirety, including remediation of all non-critical requirements which failed during examination

2. Test Information

2.1 Location

Testing will be performed at the offeror’s facility, which must be clearly identified in the offeror’s response to the RFP as requested in Section L paragraph 2.3.2.1, and Section M paragraphs 2.0 and 2.3.1.1.1 of the RFP.

The test location/room must be capable of accommodating no fewer than ten (10) government personnel, along with the simulator and required offeror personnel (in accordance with paragraph 3.2), during testing.

2.2 Equipment

The offeror must have all components of the JTC TRS System Evaluation Test configuration available for government inspection during the evaluation. This includes the simulator itself (with associated IOS, PSC), relevant technical data, EME, and any necessary support equipment required to execute effective testing. The government will provide night vision devices for tests involving the dome. Reference the JTC TRS Family of Systems (FoS) Performance Specification Appendix C, Emulated Military Equipment, for EME required during System Evaluation Test.

2.3 Configuration

The System Evaluation Test configuration for JTC TRS will be frozen at the start of testing. No modifications (hardware or software) resulting in alteration of system design or performance will be permitted during the evaluation. Correction of minor equipment malfunctions (such as routine maintenance items) which do not alter system design or performance, or prolong the test past the scheduled test period, may be allowed at the discretion of the government. Any attempt to update the system during the test period in order to improve performance may result in the proposal being eliminated from competition.

2.4 Schedule

Testing is planned to be conducted per the notional schedule outlined in Table 1, commencing no earlier than 0800 local time each day. The government reserves the right to modify the schedule as required to achieve testing objectives.

Business

Day 1 2 3 4 5

AM Event

System familiarization training, provided by the offeror for government personnel.

Scenario

Scenario

IOS / PSC

Testing

(as required)

Reserved / Supplemental

PM Event General System

Testing

EME Testing

Scenario

Scenario

Reserved / Supplemental

Reserved / Supplemental

Table 1: Notional System Evaluation Test Schedule

3. Test Methodology

3.1 Overall Approach

System evaluation testing will involve four blocks, each focused on a distinct aspect of the JTC TRS system: general system testing, EME testing, tactical scenario execution, and supplemental testing.

3.2 Test Conduct

The offeror will provide personnel (as requested by the government) necessary to operate the IOS and PSC. The offeror shall take no specific actions unless directed by the test team or granted authority to do so by the test team. The offeror shall make no inquiries as to success or failure during the evaluation, and any attempt to skew test results in their favor may result in their proposal being eliminated from competition.

The offeror’s IOS and PSC operators will each be provided a set of test cards prior to commencement of each block of testing for reference while executing the test. The set of test cards will be the same for all offerors tested. The test cards will contain specific identification of items derived from the P-Spec that correspond to the bold, underlined items provided in Attachment 1: JTC TRS System Evaluation Test Points. The test cards will be collected by the government following the completion of each test or at the end of the day.

3.3 Grading Criteria

System performance will be evaluated as pass/fail, based on the criteria described in the test points document (Attachment 1), as derived from the P-Spec. Determination of performance, as well as the need to re-execute specific scenarios or test points, remains entirely at the government’s discretion.

100% of Critical test points and 50% of Non-Critical test points shall be compliant to the P-Spec requirements to achieve a passing score.

3.4 Test Completion

Upon completion of the final test point (to the satisfaction of the government), testing will be considered complete. The offeror will not be provided real-time feedback as to system performance during the System Evaluation Test.

4. General System Test For a complete list of test points and evaluation criteria in this block, see Attachment 1:

JTC TRS System Evaluation Test Points.

5. Emulated Military Equipment Test Testing will focus on verifying the quantity, baseline functionality, and proper emulation, simulation, or stimulation of each EME article, as prescribed in Appendix C: Equipment Requirements of the JTC TRS Performance Specification. It will include hands-on evaluations of each individual item by government Subject Matter Experts (SMEs), as well as assessments of interconnectivity with the IOS and proper tracking within the domed simulator environment.

EME will be evaluated further (as required) during the operational scenarios described in the following section.

For a complete list of test points and evaluation criteria in this block, see Attachment 1:

JTC TRS System Evaluation Test Points.

6. Operational Scenario Tests

6.1 Overview

The majority of system test will occur during the execution of four (4) distinct operational training scenarios, which will provide a look at total, integrated system capability. All major aspects of the simulator, including but not limited to visual display performance, environment/entity control, EME functionality, and IOS/PSC integration, will be assessed simultaneously, as they would be during real-world training events.

6.2 Database

All four scenarios are located in the same geographic region, and will make use of the same database. The offeror may use the Digital Terrain Elevation Data (DTED) level II imagery provided by the government to build the database for the System Evaluation Test. The offeror is responsible for generation of a simulated training environment adequate to perform the testing. The System Evaluation Test will not evaluate the quality of the database.

Location: Military Grid Reference System (MGRS): 42S WD 170 086 (approximately 15 kilometers south of Kabul, Afghanistan) Scale: Include terrain features (resolution as specified in the Statement of Work (SOW)), man-made structures, and other surroundings out to 10 kilometers Level-of-Detail: As necessary to meet the System Evaluation Test Requirements.

6.3 Scenario Summaries

Attachments 2, 3, 4, and 5 provide summaries of the four scenarios which will be used during the System Evaluation Test. In preparing each scenario, offerors should include (as entities) all ground forces outlined in the “Key Information” block, as well as surface artillery and airborne assets described in “Annex D: Assets Available.” Additional scenario information is provided as background, and the test flow may not align with that described in each specific scenario.

6.4 Scenario 1 (See Attachment 2: Scenario 1 Description)

This daytime scenario involves the neutralization of a Taliban training camp by air-dropped munitions in a moderate-density environment. Testing will include the following training tasks:

Initial Aircraft Check-In / JTAC Situation Update JTAC Visual Orientation (Aided/Unaided) Target Identification / Designation Aircraft De-confliction from Surface Artillery Target Attack (F-16) Mission Debrief

For a complete list of test points and evaluation criteria for this scenario, see Attachment 1: JTC TRS System Evaluation Test Points.

6.5 Scenario 2 (See Attachment 3: Scenario 2 Description)

This night scenario involves the neutralization of an Improvised Explosive Device (IED) production facility by airborne munitions in a moderate-density environment. Testing will include the following training tasks:

Night Visual Orientation (Aided/Unaided) Staggered Check in of Multiple Close Air Support (CAS) Assets Forward-Looking Infrared (FLIR) Sensor Talk-On Remotely-Operated Video-

Enhanced Receiver (ROVER) Target Attack (F-15E) Infrared (IR) Marker / IR Zoom Laser Illuminator Designator (IZLID)

Employment Target Attack (MQ-1) Aircraft Entity Management Ground-Based Illumination Target Attack (AC-130)

6.6 Scenario 3 (See Attachment 4: Scenario 3 Description)

This day scenario involves support of coalition ground forces, as they progress through a dislocated, high-density environment and come under coordinated enemy fire.

Testing will include the following training tasks:

Friendly Forces / Joint Fires Observer (JFO) Check-In Target Mark (AH-64) Target Attack (MQ-1) Trainee Relocation to Urban Environment and Vehicle / Transition to Night Target Attack (Urban) Target Illumination / Attack (AH-64)

6.7 Scenario 4 (See Attachment 5: Scenario 4 Description)

This day scenario involves adverse weather, Suppression of Enemy Air Defenses (SEAD), and support of coalition ground forces under attack from coordinated enemy fire. Testing will include the following training tasks:

Adverse Weather PSC Control of Enemy Entities Surface-to-Air Threats Target Attack - SEAD/Artillery Target Attack - CAS Target Attack (A-10)

7. Supplemental Test Testing will focus on completing assessments of the IOS, PSC, Image Generator (IG), Environment Generator (EG), and other components.

For a complete list of test points and evaluation criteria for this block, see Attachment 1:

JTC TRS System Evaluation Test Points.

ATTACHMENTS

Attachment 1: JTC TRS System Evaluation Test Points Attachment 2: Scenario 1 Description Attachment 3: Scenario 2 Description Attachment 4: Scenario 3 Description Attachment 5: Scenario 4 Description Attachment 6: System Evaluation Test EME List Attachment 7: System Evaluation Test Model-Ordnance-Entity List

ACRONYMNS

Term Description

CAS Close Air Support DTED Digital Terrain Elevation Data EG Environment Generator EME Emulated Military Equipment FLIR Forward Looking Infrared IED Improvised Explosive Device IG Image Generator IOS Instructor/Operator Station IR Infrared IZLID Infrared Zoom Laser Illuminator Designator JFO Joint Fires Observer JTAC Joint Terminal Attack Controller JTC TRS Joint Terminal Control Training and Rehearsal System MGRS Military Grid Reference System PSC Pilot Station Console P-Spec Performance Specification RFP Request for Proposal ROVER Remotely Operated Video Enhanced Receiver SEAD Suppression of Enemy Air Defense SME Subject Matter Expert SOW Statement Of Work