Attachment_(15)_LVCMS_RACT3_TEMP_PDX121613_25_Sept_2013.docx

DOCX document 212 KB Posted

Attached to
LIVE, VIRTUAL, CONSTRUCTIVE MODELING & SIMULATION (LVCMS) ANTI-SUBMARINE WARFARE VIRTUAL AT-SEA TRAINING SYSTEMS (ASW VAST)
Solicitation number
N6134013R1057
Issued by

About this file

This file is the RACT 3 TEMP added to the solicitation under attachment 15.

View the file

Other files

Show all 11

On GovTribe

Work with this file

  • Download the original file
  • Contacts named in this file
  • Similar government files
  • Ask GovTribe AI about this file

Text version

TEMP PDX121613Rev: -
9 September 2013

TEST AND EVALUATION MASTER PLAN

FOR THE

LIVE, VIRTUAL, CONSTRUCTIVE MODELING & SIMULATION (LVCMS)

ANTI-SUBMARINE WARFARE VIRTUAL AT-SEA TRAINING SYSTEMS (ASW VAST)

NAVAL AIR WARFARE CENTER

TRAINING SYSTEMS DIVISION

12350 RESEARCH PARKWAY

ORLANDO, FLORIDA 32826-3275

DISTRIBUTION STATEMENT D - Distribution authorized to the Department of Defense and U.S. DoD contractors only for Administrative or Operational Use (determined 30 January 2013). Other requests for this document shall be referred to Commander, Naval Air Systems Command (PMA205), 47123 Buse Road, Patuxent River, MD 20670.

WARNING - This document contains technical data whose export is restricted by the Arms Export Control Act (Title 22, U.S.C., Sec 2751, et seq.) or the Export Administration Act of 1979 (Title 50, U.S.C., App. 2401 et seq.), as amended. Violations of these export laws are subject to severe criminal penalties. Disseminate in accordance with provisions of DoD Directive 5230.25.

DESTRUCTION NOTICE - For unclassified, limited distribution documents, destroy by any method that will prevent disclosure of contents or reconstruction of the document xviii xviii

TEMP REVISION HISTORY AND APPROVAL PAGE(s)

REV

LTR

REVISION
DESCRIPTION
AFFECTED AREAS
DATE
-
Preliminary Release

9 Sept 2013

TEMP PDX121613
Rev: -
9 September 2013

ii Table of Contents

Section/Para.TitlePage
PART IINTRODUCTION1
1.1Purpose1
1.2Mission Description1
1.3System Description1
1.3.1System Threat Assessment1
1.3.2Program Background1
1.3.2.1Previous Testing2
1.3.3Key Capabilities2
1.3.3.1Key Interfaces2
1.3.3.2External Interfaces with Existing or Planned Systems.2
1.3.3.3Special Test or Certification Requirements2
1.3.3.4Systems Engineering Requirements2
PART IITEST PROGRAM MANAGEMENT AND SCHEDULE2
2.1T&E Management2
2.1.1T&E Organization Construct4
2.1.2Government4
2.1.2.1NAVAIR PMA-205– Sponsoring agencies4
2.1.2.2NAWCTSD Orlando4
2.1.2.3Fleet Project Team4
2.1.2.4Trainer Custodian4
2.1.3Contractor5
2.1.3.1Prime Contractor5
2.1.3.2Supporting Subcontractor(s)5
2.1.4Integrated Test Team5
2.2Common T&E Database Requirements6
2.3Deficiency Reporting6
2.3.1DR Categories6
2.3.2Deficiency Report Review Board (DRRB)6
2.3.3DR Sign-off6
2.3.4Government Retesting of Corrected Deficiencies7
2.3.5Regression Testing7
2.3.6Testing Stoppage7
2.3.7Testing Termination7
2.3.8Test Log7
2.3.9Trainer Test Procedures (TTP)8
2.3.10Test Inspection Report (TIR)8
2.4TEMP Updates8
2.5Integrated Test Program Schedule8
PART IIITEST AND EVALUATION STRATEGY9
3.1T&E Strategy9
3.1.1In-process inspections10
3.1.2DT-1 (CPI)10
3.1.3Test Readiness Review (TRR)10
3.1.4DT-2 (GPI)10
3.1.5NAWCTSD Laboratory Integration11
3.1.6DT-5 (JFI)11
3.1.7PCA11
3.1.7.1PCA Lockout-tagout procedures12
3.2Evaluation Framework12
3.3Developmental Evaluation Approach12
3.3.1Mission Oriented Approach12
3.3.2Developmental Test Objectives12
3.3.3Modeling & Simulation12
3.3.4Test Limitations12
3.3.5Live Fire Test & Evaluation Approach13
3.4Certification for Initial Operational Test & Evaluation13
3.5Operational Evaluation Approach13
3.6Other Certifications13
3.7Reliability Growth13
3.8Future Test and Evaluation13
PART IVRESOURCE SUMMARY13
4.1Introduction13
4.1.1Test Articles13
4.1.2Test Sites and Instrumentation13
4.1.3Test Support Equipment13
4.1.4Threat Representation13
4.1.5Test Targets and Expendables14
4.1.6Operational Force Test Support14
4.1.7Models, Simulations and Test Beds14
4.1.8Joint Mission Environment14
4.1.9Special Requirements14
4.2Federal, State and Local Requirements14
4.3Manpower/Personnel Training14
4.4Test Funding Summary14

iii

LIST OF TABLES

TableTitlePage
TABLE I.Integrated Test Program Schedule8

LIST OF APPENDICES

AppendixTitlePage
ABibliography15
BAcronyms16
CPoints of Contacts18

PART I INTRODUCTION

Purpose This new Test and Evaluation Master Plan (TEMP) is intended to document the overall structure and objectives of the Test & Evaluation (T&E) program for the Live, Virtual, Constructive Modeling & Simulation (LVCMS) Anti-Submarine Warfare Virtual At-Sea Training Systems (ASW VAST) MH-60R Air Crew Tactical Team Trainer (RACT3) training system. This is an initial TEMP which will be continuously reassessed and refined as trainer development, production, and testing progress to include changes in T&E concepts, test responsibilities, mission, systems descriptions, T&E ground rules, schedules, documentation, and resource requirements. This TEMP will be an agenda item at each scheduled requirements, design, progress, and test review. As used in this TEMP, the terms training system, training device, trainer, and RACT3 are equivalent.

Mission Description The primary use of the RACT3 training system is to provide an integrated training capability that supports the MH-60R community in Anti-Submarine Warfare (ASW) proficiency.

System Description The MH-60R Air Crew Tactical Team Trainer (RACT3) prototype training device will provide active, passive and multi-static acoustics training in a deployable PC based simulation for the MH-60R ASW tactical nucleus (Pilot, Co-Pilot, and Sensor Operator) and is also configured with an Instructor Operator Station (IOS). The RACT3 will be part of the LCVMS and ASW VAST family of training systems that are networked, PC-based deployable trainers designed to support integrated and coordinated ASW, Anti-Access Area-Denial (A2AD) and Cyber Warfare tactical training using Joint Semi Automated Forces (JSAF) Navy Training Baseline (NTB) simulation. The RACT3 training system may be located in various locations, locally, nationally, and/or internationally.

System Threat Assessment The trainer will provide a level of security (physical and information) that is commensurate with the security level of the information handled by the trainer. The trainer will meet the Information Assurance (IA) criteria specified in Department of Defense Instruction (DoDI) 8500.2 for a Mission Assurance Category (MAC) III, CLASSIFIED Confidentiality Level. The trainer will meet the Information Assurance (IA) criteria specified in Naval Air Warfare Center Training Systems Division (NAWCTSD) SOW PDX13EE03 or SOW PDX13EE04.

Program Background The LVCMS and ASW VAST family of training systems use Commercial Off-the-Shelf (COTS) hardware, Linux and Windows operating systems, common ASW training software products, and trainer-unique software to emulate, simulate, and/or stimulate various Naval tactical systems for training purposes. The LCVMS and ASW VAST family of training systems are networked, PC-based deployable trainers designed to support integrated and coordinated ASW, Anti-Access Area-Denial (A2AD) and Cyber Warfare tactical training using Joint Semi Automated Forces (JSAF) Navy Training Baseline (NTB) simulation. The current training system configuration is comprised of personal computers (PC), each PC system representing a mission training station in a particular aircraft (i.e. SH-60B, SH-60F, P-3C) or Ship-board system which, when in a networked environment, allow for integrated mission training. The Navy has requirements for the design, fabrication, test, development and lifecycle modification of the LVCMS ASW VAST training systems to include aircraft, surface, sub surface ships or combat systems which, when in a networked environment, allow for integrated mission training. The MH-60R Air Crew Tactical Team Trainer (RACT3) prototype training device will be the latest training device to be built as part of this family of training systems.

Previous Testing Not Applicable Key Capabilities There are no Key Performance Parameters (KPPs) or Key System Attributes (KSAs) for this effort.

Key Interfaces External Interfaces with Existing or Planned Systems.

The RACT3 will be standalone prototype trainer with the capability to be network connected both locally at each site and over an external distributed network for training. The RACT3 will be interoperable with the Navy Continuous Training Environment (NCTE). The RACT3 will interoperate with Distributed Interactive Simulations (DIS) and High Level Architecture (HLA) Federation networks configured to support a training event.

Special Test or Certification Requirements The trainer will require Information Assurance (IA) tests for Certification & Accreditation (C&A). The trainer will be required to implement the necessary security measures to obtain the Authority of Operate (ATO).

Systems Engineering Requirements The System Engineering Technical Reviews (SETRs) will be conducted on the RACT3 in accordance with NAVAIRINST 4355.19D.

PART II TEST PROGRAM MANAGEMENT AND SCHEDULE

T&E Management PMA-205 is the cognizant program office for the RACT3 and is the program sponsor. PMA-205 is the Program Manager for the RACT3 devices and has overall program responsibility and is responsible for the Test & Evaluation for the program. The government Test Director will chair the T&E Integrated Product Team (IPT). The T&E IPT will be formed early in the trainer development program and will consist of individuals from the procuring activity, Government Subject Matter Experts (SME), Fleet Project Team (FPT), sponsoring agency, trainer custodian, other Government agencies, and trainer contractor and subcontractor’s test personnel. The T&E IPT will focus on events leading up to, and including Developmental Testing (DT) and Test Readiness Review (TRR). There is no requirement for formal Operational Test (OT). The roles and responsibilities of all participating organizations are described below:

A. NAVAIR PMA-205, Patuxent River, MD:

(1) Provide necessary funding resources to the test and support agencies conducting test and evaluation.

B. NAWCTSD Lead Engineer, Orlando, FL:

(1) Serve as T&E IPT Chair

(2) Coordinate test and evaluation requirements with appropriate test and support agencies, prepare necessary agreements.

(3) Coordinate the T&E strategy as outlined in the TEMP.

(4) Provide T&E guidance in support of SETR events.

(5) Assign Test Director (TD) or on-site TD representative.

C. NAWCTSD Engineer, Orlando, FL:

(1) Provide technical support at T&E Working groups, and risk management boards. Provide inputs to Test Plans and assist in the T&E test strategy.

(2) Provided technical support as a member of the test and evaluation team to include on-site testing and test results analysis.

(3) Review and provide recommendations for approval of all contractor-prepared test plans, procedures, and reports.

(4) Monitor and review results of Developmental Testing.

(5) Support planning, conduct, analysis, and reporting of Test Phases.

(6) Function as or coordinate with on-site TD representative.

(7) Evaluate training and documentation materials.

(8) Provide progress reports as necessary during Test Phases.

(9) Identify deficiencies and generate Deficiency Reports (DRs) as appropriate.

(10) Prepare & provide test reports as necessary.

D. Fleet Project Teams

(1) Provide Subject Matter Expert (SME) support to the Test phases as needed.

(2) Provide support to the T&E IPT as needed to develop mission based testing scenarios and provide SME support for training and readiness requirements.

(3) Attend meetings and program reviews as appropriate.

E. Contractor

(1) Upon contract award the industry team will provide engineering, design, fabrication, program management, logistics, installation, and test and evaluation services and support as listed in the LVCMS ASW VAST SOW PDX13EE03.

T&E Organization Construct There will be T&E IPT meetings chaired by the Government Test Director that are scheduled to coincide with the regularly scheduled meetings, such as System Requirements, Design, Progress, and Test Readiness Reviews. T&E IPT meetings shall be held through the length of the contract as needed, most likely with frequency increasing when approaching test phases. Responsibilities of key individual T&E IPT members are provided in Section 2.1 above.

Government NAVAIR PMA-205– Sponsoring agencies The sponsoring agencies, PMA 205 will participate in the T&E IPT as needed.

Function
Name, Office Code
Telephone No.
PMA 205 Representative
TBD
TBD

NAWCTSD Orlando Members will include, but not be limited to, the following:

Function
Name, Office Code
Telephone No.
Test Director (TD)
TBD
TBD
Lead LVCMS Lab Engineer
Jacob Bourque
407-380-4753
Software Engineer
Scott Stallard
407-380-8575
In-Service Engineer (various)
TBD
TBD
Information Assurance Officer
Joe Maldonado, 7.2.6
407-380-8209

Fleet Project Team Specific SMEs will be identified as needed.

Function
Name, Office Code
Telephone No.
Fleet Project Team Representative
TBD
TBD

Trainer Custodian Trainer custodian personnel will participate in the T&E IPT as needed.

Function
Name, Office Code
Telephone No.
Trainer Custodian
TBD
TBD

Contractor Prime Contractor The contractor’s T&E IPT members will include, but won’t be limited to, the following:

Function
Name, Office Code
Telephone No.

Test Manager

Test Engineer

Software Engineer

Subject Mater Experts (SMEs)

Supporting Subcontractor(s) Supporting subcontractor’s T&E IPT members will include, but won’t be limited to, the following:

Subcontractor’s Name
Function
Name
Telephone No.

Integrated Test Team Integrated Test Team members from the T&E WIPT are identified below:

Function
Organization/Company
Telephone No.
Test Director (TD)
NAWCTSD
Test Manager
Contractor
Lead LVCMS Lab Engineer
Jacob Bourque, NAWCTSD
407-380-4753
Software Engineer
Scott Stallard, NAWCTSD
407-380-8575
Software Engineer
Contractor
Information Assurance Officer
Joe Maldonado, NAWCTSD
407-380-8209

Common T&E Database Requirements All T&E products and technical data will be in digital format. Technical data includes the deficiency database, TTP, T&E data such as presentations, meeting notes, generated data items, specifications, RTVM, drawings and action items/trouble reports.

Deficiency Reporting Deficiency Reports (DR) will be used to identify problems related to validating that the system meets the mission of the training system. A deficiency reporting system will be used for tracking (identification, assigned, status, progress, resolution) hardware and software problems (including tactical subsystem) discovered during testing. In addition, the deficiency reporting system will track documentation problems including technical manual and data errors. The contractor will provide the government with access to the contractor's deficiency tracking system. The follow deficiency definitions and descriptions apply.

DR Categories DRs shall be categorized as Part I through Part III. During the Conformance Inspections, the Government will assign DR categories as deemed appropriate. The following are the definitions of the DR categories:

a. Part I* (safety/critical) - Jeopardizes safety. Deficiency results in safety hazard(s) to personnel and/or equipment. Testing shall not proceed until the safety hazard is satisfactorily resolved, as determined by the Government.

b. Part I (critical) - Prevents the accomplishment of an operational or mission essential training system capability, or jeopardizes security, life cycle support, or other critical requirements, as determined by the Government. Deficiency results in significant degradation in mission accomplishment, technical performance, or supportability. Acceptable workarounds may not be available or may have negative consequences, as determined by the Government.

c. Part II (major) - Adversely affects the accomplishment of an operational or mission essential training system capability, life cycle support, or other critical requirements, as determined by the Government. Deficiency results in significant degradation in mission accomplishment, technical performance or reliability and maintainability, but a work-around solution is available and potentially acceptable, as determined by the Government.

d. Part III (minor) - Minor reduction in technical performance or supportability, but can be tolerated with little or no impact on mission accomplishment.

Deficiency Report Review Board (DRRB) The DRRB membership will include Government and Contractor test team personnel. The DRRB members and processes will be determined after contract award as part of the T&E IPT meetings. The DRRB will review and prioritize and classify DRs identified during test events. The results will be listed in the DR database.

DR Sign-off When corrected, each deficiency will be signed-off by its originator or the originator’s designated representative and the TD or the TD’s designated representative.

Government Retesting of Corrected Deficiencies The Government reserves the right to repeat any and all tests for the purpose of verifying the effectiveness of a deficiency correction.

Regression Testing When there is a need to verify that a change made to the trainer hardware or software during the T&E phase has not inadvertently affected trainer functionality, regression testing will be performed in accordance with an acceptable test series as determined by the Government. As a minimum, regression testing will be performed:

a. When a new software load is installed.

b. When verifying DR corrections.

c. When the trainer load/disks are rebuilt from source code (cold start).

d. When reverting to a previous baseline.

e. When a hardware change has the potential to affect complex interfaces in an unpredictable way.

f. When the Government needs to verify that the trainer baseline has not been corrupted.

Testing Stoppage Testing can be stopped by anyone at any time for safety concerns of the system. If a safety concern is raised during testing all personnel should immediately cease testing and report safety concerns to the TD and contractor test team.

Testing Termination The TD has the authority to terminate a test phase if the severity of the DRs makes the system unusable/un-testable and/or inoperative trainer systems and subsystems. The duration of test phase termination will be at the discretion of the TD, but will be discussed within the T&E IPT. Unlike test stoppage, termination refers to the complete cessation of Test Procedure execution based on issues that affect many procedures. The TD should use the following guidelines:

1. Part I DR that is terminal: Does the deficiency preclude further testing of the trainer such that the problem must be corrected prior to continuation of testing?

2. Numerous Part I and/or Part II DRs: Does the accumulation of major deficiencies affect functionality in numerous areas of the trainer? Do the problems interrupt execution of the tests to the point that time would be better spent evaluating trainer issues than to continue testing?

3. Excessive Total DRs: Does the total quantity of all DRs (Parts I/II/III) degrade trainer operation to the extent that testing or training cannot proceed uninterrupted without an excessive amount of workarounds?

Test Log The RACT3 trainer test log will be the primary record for all testing and modifications performed on the test article. The log will be a bound book with page numbers that will be available at all times during the Trainer acceptance test period. All personnel testing or modifying the Trainer will be responsible for documenting all actions in the Test Log. If any modifications are made to Trainer hardware or software, then it is the responsibility of the individual making those changes to record that event in the log. An entry into the log will contain the date, time, name of individual, and a description of the Trainer-related activity.

Trainer Test Procedures (TTP) The Test Procedures are a CDRL item that is prepared/updated by the Contractor and reviewed by the Government for acceptance. The TTP provides detailed test procedures which will be used to test subsystem functionality as it pertains to meeting each Delivery Order (DO) requirements and derived requirements of the contract. In addition, the TTP will be used to test the complete system as it pertains to meeting the overarching system level requirements. Test results will be documented in the TIR in accordance with the CDRL.

Test Inspection Report (TIR) Integrated test results will be documented in the Test Inspection Report (TIR). The TIR is a CDRL item that is prepared by the Contractor and reviewed by the Government for acceptance. The TIR documents the results of testing in accordance with the CDRL requirements.

TEMP Updates This is the initial draft TEMP. Planned updates will occur as needed and IAW the TEMP CDRL in SOW PDX13EE03. IPT meetings will be held as necessary to provide continuous feedback for TEMP updates.

Integrated Test Program Schedule The generic integrated test program schedule is included in the overall program schedule that is shown in TABLE I. Major program milestones include SETR events are also shown in TABLE I, with dates to be identified at each DO contract award and continuously updated throughout the period of performance.

TABLE I.Integrated Test Program Schedule
PROGRAM MILESTONES
Date
Contract Award
TBD
Post Award Conference
TBD
System Requirements Review/System Functional Review (SRR/SFR)
TBD
Preliminary Design Review (PDR)
TBD
Critical Design Review (CDR)
TBD
In-process Inspections
TBD
Contractor Preliminary Inspection (CPI) (Developmental Testing, DT-1)
TBD
Test Readiness Review (TRR)
TBD
Government Preliminary Inspection (GPI) (Developmental Testing, DT-2)
TBD
*Contractor Final Inspection (CFI) (Developmental Testing, DT-3)
N/A
*Government Final Inspection (GFI) (Developmental Testing, DT-4)
N/A
*NAWCTSD Lab Integration
TBD
*Joint Final Inspection (JFI) (Developmental Testing, DT-5)
TBD
Physical Configuration Audit (PCA)
TBD

* NOTE – The IPT will not conduct separate CFI & GFI for the RACT3 device. CFI and GFI will be combined into a Joint Final Inspection (JFI) (Developmental Testing, DT-5), as required in SOW PDX13EE03.

PART III TEST AND EVALUATION STRATEGY

T&E Strategy The T&E strategy is to demonstrate that the trainer complies with the performance requirements specified in each DO and also meets the user requirements to fulfill their training requirements. The test team will accomplish this strategy by testing each sub-system and the system as a whole during different phases of system maturation to identify potential risks and to assess the system’s potential for meeting both the contract and user requirements as early as possible in the development cycle of the RACT3 training device. The following is a chronological outline of the trainer T&E process. Please refer to SOW PDX13EE03 for the specific T&E program requirements. The objectives of each of the following T&E program elements are described in greater detail below:

a) In-process Inspection

b) DT-1

c) DT-2

d) Test Readiness Review (TRR)

e) DT-3 (N/A)

f) DT-4 (N/A)

g) NAWCTSD Lab Integration

h) DT-5 (combined DT-3 & DT-4)

i) PCA In-process inspections The contractor will perform and the government may monitor the In-process inspections that will include visual, electrical, and mechanical examinations and testing of materials, subassemblies, parts, and accessories (including purchased items), prior to integration into the end item that may be required to assure conformance to all requirements of the contract.

DT-1 (CPI)

The contractor will perform DT-1 in-plant testing IAW government-approved test plans. The contractor will conduct the tests incrementally under the direction of the contractor’s Quality Management System (QMS) representative. DT-1 test results may be monitored by the government Test Engineering representative. The contractor will annotate in the TTP changes made as a result of DT-1 and shall provide a copy of the annotated TTP to the Government prior to the TRR. The contractor will document the DT-1 results in the Test Inspection Report. The contractor will record tests results that do not comply with the DO requirements. The contractor will correct the deficiencies found during DT-1 prior to TRR. The documented DT-1 results, including open deficiencies, will be presented to the Government on an incremental basis at the next scheduled T&E IPT meeting and in final total form at the TRR.

Test Readiness Review (TRR) TRR will be conducted following successful completion of contractor DT-1 to determine trainer readiness for DT-2. Please refer to SOW PDX13EE03 for the specific review items to be covered in the TRR, the TRR entry criteria, and the TRR exit criteria. TRR will be conducted by the Government with Contractor support prior to DT-2 IAW Systems Engineering Technical Review Process, NAVAIRINST 4355.19D. The government may tailor the TRR based on the technical scope and risk of the effort. The contractor will provide the resources, including facilities, equipment and personnel to support the TRR. The TRR will include a review of the T&E program, including the test results, presentation of contractor certification of test readiness and open deficiencies remaining from DT-1. The TRR will assess test objectives, test methods and procedures, scope of tests, and determines if required test resources have been properly identified and coordinated to support planned tests.

DT-2 (GPI)

DT-2 will commence after TRR has been successfully completed. DT-2 will be conducted in-plant by the Government test team. DT-2 will consist of Government-conducted tests to demonstrate compliance with the specified performance requirements and user requirements. DT-2 will be conducted IAW the Government-accepted TTP and other Government-accepted test plans. The contractor will provide the Government with a copy of the revised TTP prior to DT-2. The Government reserves the right to perform additional tests to ensure compliance with the specified requirements. Testing will commence with the establishment of a software baseline resulting from a software cold-start. The Government test team will perform the IA verifications IAW the verification requirements of SOW PDX13EE03, and the Government-accepted TTP. The contractor will provide the resources, equipment, and facilities needed to support the tests. Personnel identified by the T&E IPT will support the testing. Hardware and software configuration item testing and design documentation verification and validation will be conducted as an integral part of DT-2. Deficiencies found during these tests shall be documented as described above and adjudicated by the DRRB. DT-2 results will be recorded in the TIR in accordance with the CDRL.

NAWCTSD Lab Integration Contractor’s hardware and software will be delivered to NAWC TSD to be integrated with NAWCTSD software and hardware prior to shipment to the training site for final installation and JFI.

DT-5 (JFI)

DT-5 will commence when the DT-2 exit criteria have been met and integration with NAWCTSD hardware and software has been completed. DT-5 will be conducted on-site by a contractor and government test team and will consist of tests conducted to verify compliance with the specified DO requirements. DT-5 will be conducted IAW the Government-accepted TTP (revised to include corrections made during previous test periods) and other Government-accepted test plans. The contractor will provide the Government with a copy of this revised TTP prior to DT-5. Testing will commence with the establishment of a software baseline resulting from a software cold-start, the verification requirements of the DO and the TTP. Following the cold-start, the test team will perform the IA verifications IAW the verification requirements of DO and the TTP. Deficiency correction verification and validation, including additional cold-starts, will be at the discretion of the Government. The contractor will provide the resources, and equipment necessary to support the tests. Personnel identified by the T&E IPT will support the testing. Hardware and software configuration item testing and design documentation verification and validation will be conducted as an integral part of DT-5. Deficiencies found during DT-5 shall be corrected by the contractor and verified by the test team prior to Government acceptance of the trainer.

PCA

The PCA will consist of non-functional examinations performed to demonstrate that the trainer as-built design satisfies SOW PDX13EE03 requirements, and that the deliverable hardware and software documentation accurately reflect the configuration items. The PCA will consist of software and hardware examinations defined in the Government-accepted TTP. The hardware portion of the PCA will consist of an examination of the as-built trainer against its design documentation. The software portion of the PCA will consist of an examination of the as-built version of the computer system configuration items against the software technical documentation. The PCA shall be conducted to the extent specified in SOW PDX13EE03. The PCA will include following components:

a. PCA Kickoff Meeting - Before the PCA examinations commence, the Government ILSM (in conjunction with the contractor co-chairman) should convene a kickoff meeting, with all Government and contractor participants, to address ground rules to be followed during the PCA.

b. Engineering Design Materials Review - During the PCA, engineering design materials will be reviewed for format. Future revisions of the TEMP will list the specific areas to be reviewed.

c. Hardware Audit - During the PCA, the trainer hardware will be examined for compliance with contract requirements. Future revisions of the TEMP will list the specific areas to be reviewed.

d. Technical Documentation (TD) Audit - During the PCA, the trainer TD will be examined. Future revisions of the TTEP will list the specific areas to be reviewed.

e. PCA close out meeting - Following the review of all hardware and documentation, a final meeting of the PCA team should be convened. Future revision of the TEMP will list the specific items to be covered during the PCA close out meeting.

PCA Lockout-tagout procedures Prior to the start of the PCA examinations, the contractor will perform a lockout-tagout of the trainer equipment and present a written certification certifying that all electrical power, including UPS systems, have been disconnected, that the training system has been de-energized, and that the pneumatic and hydraulic system pressures have been released. Future revisions of this TEMP will describe the specific lockout-tagout procedure to be used.

Evaluation Framework The Test Phases (DT-1, DT-2 & DT-5), TRR, and PCA will be used to support the acceptance decision of the device.

Developmental Evaluation Approach Mission Oriented Approach The test procedures will be evaluated during the test phases with the help of fleet SMEs to be representative of how the training device will be used in normal day-to-day operation.

Developmental Test Objectives The objective of the T&E program is to demonstrate that the trainer complies with the performance requirements specified in SOW PDX13EE03.

Modeling & Simulation There are no planned modeling and simulation activities that will be used for test and evaluation. All tests and demonstrations will be conducted directly on the training device.

Test Limitations Future revisions of this TEMP will discuss any additional test limitations that may significantly affect the T&E IPT’s ability to draw conclusions, the impact of these limitations, and resolution approaches. At the revision date of this TEMP, there are no known limitations.

Live Fire Test & Evaluation Approach This program does not fall under the provisions of United States Code Title 10, Section 2366, Live Fire Test & Evaluation (LFT&E), therefore no LFT&E is required.

Certification for Initial Operational Test & Evaluation There is no requirement for formal Operational Test (OT) for this effort.

Operational Evaluation Approach There is no requirement for formal Operational Test (OT) for this effort.

Other Certifications No other certifications are required for this effort.

Reliability Growth Reliability Growth is not a requirement for this effort.

Future Test and Evaluation No future test and evaluation is planned for the effort.

PART IV RESOURCE SUMMARY

Introduction Test Articles In this section, future revisions of this TEMP will identify the actual number of and schedule requirements for all test articles, including key support equipment and technical information required for testing in each phase of the T&E program. It will identify if key subsystems (components, assemblies, subassemblies or software modules) are to be tested individually, before being tested in the final system configuration, each subsystem and the quantity required. It will also identify when prototype, engineering development, pre-production, or production models will be used during testing.

Test Sites and Instrumentation Testing and Evaluation as directed per this document shall occur at NAWC TSD, Orlando, Florida.

Test Support Equipment In this section, future revisions of this TEMP will identify test support equipment that must be acquired specifically to conduct the test program. It will clearly identify who is responsible to provide it.

Threat Representation The RACT3 will require artificial threats as part of Test and Evaluation. These artificial threats are provided by the JSAF that resides on the trainer.

Test Targets and Expendables No test targets or expendables are applicable.

Operational Force Test Support No support from Operational Forces is required.

Models, Simulations and Test Beds Not applicable for this effort.

Joint Mission Environment No testing in a Joint Mission Environment is required for this effort.

Special Requirements There are no special requirements to execute the Test Phases in this TEMP.

Federal, State and Local Requirements Testing and Evaluation directed per this TEMP will not produce any products or materials at risk of not complying with Federal, State, and Local Requirements. Testing and Evaluation shall comply with OSHA 29 CFR Part 1910.

Manpower/Personnel Training Prior to commencement of Testing and Evaluation, the contractor shall provide training to Naval Instructors and NAVAIR personnel detailing the operation of the RACT3, if needed.

Test Funding Summary Not applicable for this effort.

Appendix A

Bibliography References Program unique documents The following Program-Unique documents are referenced in this TEMP:

NAWCTSD

SOW PDX13EE03
-
Sample 1, Statement of Work For LVCMS ASW VAST, dated 9 September 2013

Program-generated T&E documents

a. Test Procedure

b. Test/Inspection Report

c. TRR agenda and minutes

d. Progress and design review agenda and minutes sections related to T&E issues

e. Testing Logs

f. Deficiency Logs

g. Test Deficiency Reports

h. IA Test Report

i. DIACAP C&A Package

j. Software/Firmware Change Request

k. Trainer Engineering Report

l. Training System Support Document (TSSD)

m. Others Operational equipment documentation

a. Aircraft Naval Air Training and Operating Procedures Standardization (NATOPS) operators manual

b. Handbooks of Maintenance Instructions

c. Others Commercial equipment documentation Not applicable.

Other documentation categories Add as required.

Appendix B

Acronyms

AcronymDefinition
AOPAircraft Operating Program
ATOAuthority to Operate
C&ACertification and Accreditation
CDRCritical Design Review
CDRLContract Data Requirements List
CFRCode of Federal Regulations
COICritical Operational Issue
COTSCommercial Off The Shelf
CTTPCritical Technical Parameter
DoDDepartment of Defense
DoDIDepartment of Defense Instruction
DRDeficiency Report
DRRBDeficiency Report Review Board
DTDevelopmental Testing
E3Electromagnetic Environmental Effects
FPTFleet Project Team
GFSGovernment Furnished Software
H/SIHardware and Software Integration
IAInformation Assurance
IAWIn Accordance With
IOSInstructor-Operator Station
KTTPKey Technical Parameter
KPPKey Performance Parameter
LFT&ELive Fire Test & Evaluation
MACMission Assurance Category
MOEMeasure of Effectiveness
MOSMeasure of Suitability
NATOPSNaval Air Training and Operational Procedures Standardization
NAWCTSDNaval Air Warfare Center Training Systems Division
O&MOperation and Maintenance
OPNAVINSTOffice of the Chief of Naval Operations Instruction
OSHAOccupational Safety and Health Administration
OTOperational Test
PMAProgram Manager AIR
PDRPreliminary Design Review
SEPSystems Engineering Plan
SETRSystems Engineering Technical Review
SMESubject Matter Expert
SOWStatement of Work
SRR/SFRSystem Requirements Review/System Functional Review
TBDTo Be Determined
T&ETest and Evaluation
T&E IPTTest and Evaluation Integrated Product Team
TEMPTrainer and Evaluation Master Plan
TDTest Director
TIRTest Inspection Report
TTPTest Procedures
TRRTest Readiness Review
TSSDTraining System Support Document

Appendix C

Points of Contact

Name
T&E Function
Organization / Code
Mailing Address
Telefax Number(s)
Commercial Voice Number(s)
E-mail
DSN Voice Number(s)
Jonathan Glass
PJM
NAWCTSD

AIR 1.3.6.3

12350 Research Parkway Orlando, FL 32826

407-380-4442
407-380-8021
jonathan.glass@navy.mil
960-8021
Jacob Bourque
Lead LVCMS Lab

Engineer

NAWCTSD

AIR 4.6.8.8

12350 Research Parkway Orlando, FL 32826

407-380-4753
jacob.bourque@navy.mil
960-4753
Scott Stallard
Software Engineer
NAWCTSD

AIR 4.6.8.8

12350 Research Parkway Orlando, FL 32826

407-380-8575
scott.stallard@navy.mil
960-8575
Joe Maldonado
IAO
NAWCTSD

AIR 7.2.6

12350 Research Parkway, Orlando, FL 32826

(407) 380-8209
joseph.maldonado@navy.mil
960-8209
Robb Holmes
APJM
NAWCTSD

AIR 1.3.6.3

12350 Research Parkway Orlando, FL 32826

407-380-4442
407-380-4448
robb.holmes@navy.mil
960-4448
Marc Ennis
APJM/COR
NAWCTSD

AIR 1.3.6.3

12350 Research Parkway Orlando, FL 32826

407-380-4442
407-380-8390
marc.ennis@navy.mil
960-8390

image1.png

File details come from the government source that posted it. Updated .