Attachment 12 VLS Test Assessment Plan 11.17.21.pdf

SLS 98-1

28 APR 2016

MK 41 VERTICAL LAUNCHING SYSTEM (VLS)

TEST ASSESSMENT PLAN

PREPARED BY

NAVAL SURFACE WARFARE CENTER

CORONA DIVISION

CORONA, CA

i

List of Effective Pages

PAGE NUMBERS CHANGE IN EFFECT PAGE NUMBERS CHANGE IN EFFECT

i-iii A iv-vi Deleted

1-8 A 9-10 Deleted

Errors found in this publication, other than obvious typographical errors, which have not been corrected by means of temporary or permanent changes should be reported. Such report should include the complete title of the publication and the publication number (short title). Identify the page and line or location of the error, and forward report to:

Commanding Officer Corona Division Naval Surface Warfare Center Measurement Science and Engineering Department Test Systems Assessment Branch (MS24) PO Box 5000 Corona, CA 92878-5000 ii

Additional copies of this publication may be obtained from above addressee.

Change Record

CHANGE NUMBER DATE TITLE and/or DESCRIPTION SIGNATURE of

VALIDATING OFFICER

05/19/98

04/28/2016

Original Issue, Revision –

Revision A Major revision to incorporate VLS program changes

James Johndrow, PEO(TAD) PMS-410, Surface Launching System, Program Manager

Robert Grahovac

PEO IWS 3L

Surface Launching Systems iii

TABLE OF CONTENTS

1.0 INTRODUCTION

1.1 Purpose

1.2 Background

1.3 Policy

1.4 Scope

2.0 DEFINITIONS

3.0 CRITERIA

3.1 General

3.2 Test Configuration Baseline

3.3 Documentation

3.4 Test Performance

3.5 Measurement Capability

3.6 Calibration

3.7 Software

3.8 Test System Records

3.9 Supporting Test Disciplines

3.10 Installation and Operating Environment

3.11 Operating Personnel

4.0 ASSESSMENT AND CERTIFICATION PROCESS

4.1 Technical Data Package Review

4.2 On-site Assessment and Demonstration

4.3 Post Demonstration Analysis and Report

5.0 TYPES OF CERTIFICATION

5.1 Interim Certification

5.2 Full Certification

5.3 Renewal Certification

5.4 Special Certification

6.0 TEST SYSTEM CHANGES

1.0 INTRODUCTION

1.1 Purpose. This plan establishes the criteria, processes, and methods used to assess and certify Vertical Launching System (VLS) test capabilities. The Test Assessment and Certification program is designed to ensure that VLS test systems meet the requirements contained in applicable specifications and contracts throughout the life-cycle of the weapon system.

1.2 Background. Mission success depends, to a great extent, upon the operational readiness of weapons systems. Operational readiness can only be assured by conducting thorough and accurate testing of the weapon system and its subassemblies at production acceptance, and at other times when the condition of the weapon requires evaluation. Test assessment and certification provides a disciplined approach to ensure that test systems have the effectiveness and performance required to accurately assess the readiness of the weapon. Test assessment also reduces the Navy’s acquisition risks and costs, by providing early insight into the testing program’s ability to meet program requirements.

1.3 Policy. The Naval Surface Warfare Center (NSWC) Corona Division, Measurement Science and Engineering Department, is designated as the Test Assessment & Certification Agent for VLS, and will perform assessment and certification as detailed in this plan. As the Certification Agent, NSWC Corona shall participate in planning meetings, design reviews, technical meetings, and performance evaluations for certified test capabilities as directed by PEO IWS 3L. Table 1 contains the list of VLS test capabilities that require test assessment and certification, as of 24 March 2016.

Table 1: Certified Test Capabilities for the VLS Program

Test Capability Test System(s) CCA Test Teradyne L293/L393/Spectrum 9100 LSEQ Test LSEQ ATE/New Concept LSEQ ATE Module Test Module ATE/New Concept Module ATE MK235 MOD 3/5/6/7 Launch Control Unit (LCU) LCU Factory Acceptance Test VLS GPS Integrator (VGI) Assembly and Rack VGI Assy ATE, VGI Rack ATE VLS Canister Test MK690 MOD1/2 Canister Assy Test Set

MK708 MOD0 Canister Test Set

MK680 MOD3 CMTS

MK680 MOD4 VISTA

MK680 MOD5 CMTS

MK683 Interface Test Set MK680 MOD4 VISTA VLS Interfaces MK341 MOD0 ITS ATE Power Supply Control Module (PSCM)/ Motor Control Panel Control Module (MCM)

CAVE ATE

Motor Control Panel (MCP) MAVE ATE Programmable Power Supply (PPS) PAVE ATE Interface Convertor Ethernet (ICE) CCA, Assembly, and Canister

ICE ATE

These test capabilities must be certified prior to Government acceptance of the item being tested. Certification applies to these test systems regardless of whether the test system is Government or contractor furnished.

1.4 Scope. The general criteria and process for assessment and certification of test capabilities are contained in this document. Contract provisions, specifications, or drawings take precedence whenever a conflict exists.

2.0 DEFINITIONS

2.1 Accuracy. The degree of agreement between the true and the measured value of a quantity.

2.2 Calibration. The comparison of a measuring equipment or instrument with a National Institute of Standards and Technology (NIST) traceable standard of higher accuracy to ensure the former is within specified limits.

2.3 Correlation. The comparative analysis of test data from similar or equivalent test systems in testing like items at various locations.

2.4 Repeatability. The ability of a test system to return the same results when the same item is retested under identical conditions.

2.5 Stability. The capability of a test system to retain adjustable or defined fixed test characteristics, and to repeat measurements of non-variable values within specified tolerances over extended periods of time.

2.6 Technical Data Package (TDP). The test system baseline documentation package which includes the following elements:

(a) Unit Under Test (UUT) requirements documents such as performance specifications, product fabrication specifications, interface control drawings and acceptance test requirements.

(b) Test and calibration procedures.

(c) Specifications and programming/operating instructions for test instruments.

(d) Test system drawings including top assembly drawings, schematic diagrams, interface diagrams, and Test Equipment (TE)/UUT interconnect drawings.

(e) Test system accuracy report or measurement capability report.

(f) Software source code. This includes: application software; calibration and selftest software; fault isolation/diagnostic software; utility programs;

instrument drivers (non-Commercial Off-The-Shelf (COTS)); and configuration data files.

(g) Other supporting documentation such as Software Development Plan, Test Plan, Test Equipment Development Plan, Maintenance Manual, etc.

2.7 Test Activity. The prime contractor, subcontractor, or Government facility designated by PEO IWS 3L as the organization responsible for operating and maintaining the acceptance test system.

2.8 Test System. The functional stimuli/measurement system used to test product acceptability in accordance with established criteria. The test system is composed of the following:

(a) The test set, consisting of test equipment and instrumentation, interface devices and adapters for UUTs, ancillary hardware, and (for automatic test equipment) controlling computer, computer operating system, application programs, and applicable program libraries.

(b) Test documentation consisting of calibration and test procedures, drawings, maintenance documents, and computer software documentation.

(c) Support functions consisting of calibration/ maintenance services, configuration management, quality assurance, and access control.

(d) Operating environment and personnel, consisting of the facility and environment where the UUT testing is taking place, and the personnel who operate, calibrate, and maintain the test system.

3.0 CRITERIA

3.1 General. The criteria for test system certification are based upon (1) applicable fabrication, test, and performance specifications for the UUT, (2) contractual requirements for test equipment, and (3) the certification requirements contained in this document. The assessment and certification will include investigations into all elements of the test system defined in paragraph

2.8. The outcome of this assessment is a certification report which will provide a detailed baseline of test system capability.

3.2 Test Configuration Baseline. The test system baseline will contain, at a minimum: (1) a list of all test system documentation (defined in paragraph 2.6);

(2) a definitive list of all equipment in the system; and (3) the applicable UUT test requirements baseline.

3.3 Documentation. Documentation, whether deliverable or for contractor use only, must be adequate to: (1) evaluate test capabilities, operations, and performance; (2) verify test set configuration (hardware and software); and (3) facilitate consistent and controllable acceptance testing on a continuous basis.

All TDP documentation shall be clear, complete, correct, and conform to applicable requirements. Test requirements documents shall be traceable to higher level specifications.

3.4 Test Performance. The test system shall measure all the parameters or characteristics prescribed by applicable design, fabrication, performance, and test specifications for the UUT. This includes: (1) performing all initial conditions;

(2) applying the specified stimuli; (3) measuring the appropriate responses; (4) accurately detecting failures; and (5) collecting all required calibration, self-test, fault isolation, and acceptance test results in an appropriate format according to the contract or test plan.

3.5 Measurement Capability. The test system shall have the accuracy, range, resolution, repeatability, reliability, and stability required by applicable UUT specifications. If not specified in the UUT specification, the Test Accuracy Ratio (TAR) between the parameter being measured and the test equipment shall be a minimum of 4:1.

3.6 Calibration. The test system shall be calibrated in accordance with the following requirements:

(a) The calibration process shall have a written description containing calibration procedures, calibration intervals, sources of calibration, a list of referenced measurement standards and environmental conditions.

(b) All test equipment included in the calibration recall program shall have records of current and past calibrations. The calibration records shall include quantitative data for each calibration parameter, calibration intervals, measurement accuracies, and calibration factors or calibration curve.

(c) The measurement standards prescribed in the calibration system shall be traceable to NIST standards, and shall have a minimum TAR of 4:1 for all measurements. The standards shall also have a history of stability, and a calibration range appropriate for the parameter being calibrated.

(d) The calibration intervals shall be based on accuracy, stability, purpose, and degree of usage. Calibration periodicity is established to assure that a minimum of 85% of similar model test equipment are within tolerance when they are submitted for scheduled calibration. For one of a kind measurement and test equipment, the calibration interval is considered satisfactory if a minimum of 85% of all calibrated items for that test equipment are found to be within tolerance during calibration.

(e) The procedures for performing tests and calibration for each type of measurement and test equipment, or measurement standards requiring calibration, shall be written in a clear, concise, and step-by-step manner.

(f) Access to adjustable devices affecting the calibration or any test measurement will be sealed to deter unauthorized entry and adjustment.

Following calibration and/or maintenance, cabinets, consoles, and exposed adjustable devices will be secured and sealed with tamper-evident seals.

3.7 Software. Test software shall be written such that all test requirements are correctly implemented and data is accurately recorded. All non-COTS software shall be written in accordance with coding standards in place at the software engineering facility. Software configuration control practices shall ensure software revision control and prevent unauthorized changes to software. Only one version of acceptance software may be used for an acceptance test. If there is other software on the test system there must be positive control over which software is to be used, so that unapproved software cannot be used for acceptance testing.

All selftest, calibration and acceptance test data generated during testing must indicate the facility location, date/time, ATE model and serial number, UUT part and serial numbers, overall pass/fail status, and version of the software used during the test. Embedded UUT software shall also have the version number reported on the test report.

3.8 Test System Records. Test system records shall be maintained that include dates, down time, and description of each repair, calibration, maintenance action, and test system modification. For repairs, the type of defect, the associated repair action, and the replacement part(s) shall be recorded.

3.9 Supporting Test Disciplines. The test activity shall have the following programs documented and adhered to as required by contracts:

(a) Maintenance program.

(b) Calibration program.

(c) Configuration control program.

(d) Safety program.

(e) Quality assurance program.

3.10 Installation, Operation, and Environment. The test activity shall ensure that:

(a) Airborne particles are controlled as specified by the equipment manufacturers.

(b) Temperature and humidity are maintained within specified limits.

(c) Facility power sources (e.g., electrical, hydraulic, compressed air) meet applicable equipment manufacturer’s specifications.

(d) Hazards are identified and appropriate warnings and caution notes are provided to ensure personnel and equipment protection.

(e) Measures are in place to minimize risks of personal injury or damage to the test equipment in the event of mishap.

(f) Software was designed to prevent initiation of hazardous events or mishaps.

(g) Test equipment arrangement minimizes personal exposure to hazards during operation, calibration, maintenance, and repair. These hazards include high voltage, electromagnetic radiation (RF, laser, etc.), cutting edges, etc.

(h) Security procedures for classified testing have been evaluated by the appropriate authority when necessary.

(i) The work environment was designed to facilitate interaction between the test operator and equipment.

3.11 Operating Personnel. All operating, calibration, and maintenance personnel shall demonstrate satisfactory performance of test, maintenance, and calibration procedures during the on-site assessment.

4.0 ASSESSMENT AND CERTIFICATION PROCESS

This section identifies the assessment process that is used to establish compliance with certification requirements. The assessment and certification process will normally be performed in the sequence as outlined in this section.

4.1 Technical Data Package Review. The contractor shall submit the TDP to the Certification Agent (CA) for review at least 90 days prior to the on-site assessment and demonstration (paragraph 4.2). Contractor-internal, non-deliverable documentation shall be made available for review during the on-site assessment. The CA will review the test system documentation to verify compliance with test requirements, contracts, and certification criteria. All critical issues will be identified and recommendations for corrective actions will be provided to the contractor 30 days prior to the on-site assessment and demonstration.

4.2 On-site Assessment and Demonstration. Following completion of the TDP review, the CA will perform an on-site assessment and demonstration of the test system at the test facility. This includes, but is not limited to:

(a) Configuration audit of test system hardware, software, and documentation.

(b) Demonstration of test system calibration/selftest (One complete run performed by test activity calibration technician).

(c) Demonstration of acceptance tests (Three complete runs of the same unit to verify repeatability, performed by test activity test operator).

(d) Preliminary review of demonstration test data.

(e) Evaluation of maintenance and calibration records.

(f) Evaluation of operator personnel performance.

(g) Verification of the test activity’s quality assurance, safety, and configuration control programs.

(h) Verification of the test activity’s adherence to operational and environmental requirements of the test system.

4.3 Post Demonstration Analysis and Report. The CA will analyze the data from the on-site assessment to evaluate test system accuracy, repeatability, stability, and, for similar test systems, correlation. A final review of changes to the procedures and/or software resulting from the on-site assessment is also performed. Upon satisfactory review of all test system elements and the successful resolution of all critical issues, a formal report and certificate will be issued by the CA. This report consists of a certificate, summary of assessment results, and test system baseline including testing capability. Copies of the certificates should be kept on file by the test activity.

5.0 TYPES OF CERTIFICATION

5.1 Interim Certification. Interim certification applies to new test capabilities and test systems undergoing major changes. Interim certification is issued when the test system is deemed capable of performing the required tests, but corrective actions resulting from the certification review are not yet completed. Interim certification will normally be issued for a period not to exceed 6 months.

5.2 Full Certification. Prior to the end of the interim certification period, the CA will verify that all corrective actions resulting from initial certification have been completed. Upon successful review, full certification will be issued for the test capability. Full certification is valid until the test system has been modified, relocated, or out of service for at least six months.

5.3 Renewal Certification. Certification renewal is required when the test system has been out of service for six months or longer, or the test system is relocated to another test activity. The test system will be assessed for continued conformance to test requirements, and an on-site test demonstration in accordance with paragraph 4.2 may be required.

5.4 Special Certification. On a case-by-case, special certification may be applied to the test systems that do not conform to the defined certification criteria, yet are capable of making the required measurements. Requirements for special certification will be defined by the Navy and the CA as the situation dictates.

6.0 TEST SYSTEM CHANGES

Changes to certified test capabilities shall be submitted to the CA for review prior to being implemented and used for acceptance testing. This includes the submittal of all affected drawings, procedures, documentation, and software. If the CA determines that a change is a major modification, re-demonstration of the test system may be required. Major modifications include, but are not limited to, new ATE programming language, new UUT part number or baseline, or significant redesign of test equipment.