eCASS_FRP_2_RFP_Attachment_9_-_Reliability_and_Maintainability_Program_Plan_-_2017.06.06.pdf

REV: A

electronic Consolidated Automated Support System (eCASS)

Contract Number: N68335-17-R-0129

Document Number: eCASS-FRP-0001

Reliability and Maintainability Program Plan Date: May 18, 2017

REV: A

NAVAL AIR WARFARE CENTER AD (LKE)

CODE 2.5.2.1.3

HWY 547 BLDG 562-3 RM 310

LAKEHURST, NJ 08733-5082

DISTRIBUTION STATEMENT D. Distribution authorized to Department of Defense (DoD) and U.S. DoD contractors only for Administrative or Operational purpose, effective 15 August 2005. Other requests shall be referred to the Naval Air Systems Command (PMA260), 47123 Buse Road, IPT Building 2272, Room 349, Patuxent River, MD 20670-1547.

DESTRUCTION NOTICE - For classified documents, follow the procedures in DOD 5200.22-M, Industrial Security Manual, Section 11-19 or DOD 5200.1-R, Information Security Program Regulation, Chapter IX. For unclassified, limited documents, destroy by any method that will prevent disclosure of contents or reconstruction of the document.

eCASS RELIABILITY AND MAINTAINABILITY PROGRAM PLAN eCASS-FRP-0001

REV A

II

CHANGE HISTORY

LIST OF AFFECTED PARAGRAPHS

This listing contains the issue dates of all revisions to this document. Revised sections shall be placed in this manual and this listing updated to replace the existing List of Affected Paragraphs.

Paragraph No Rev./Date Change Description

All 03/18/2017 New Release

All 05/18/2017 Prepared for Government release

REV A

III

Table of Contents

1 INTRODUCTION

1.1 SCOPE

1.2 PURPOSE

1.3 REFERENCE DOCUMENTS

2 GENERAL

2.1 ORGANIZATION

2.2 FUNCTIONS

2.2.1 Reliability Tasks

2.2.2 Maintainability Tasks

2.2.3 Support Tasks

2.3 DUTIES

2.3.1 Reliability and Maintainability Engineering

2.3.2 Technical Reviews

2.4 RESPONSIBILITIES

2.4.1 Reliability and Maintainability Program Integration

2.4.2 Maintainability Interface Compatibility

2.4.3 Failure Review Board (FRB)

2.5 REPORTING REQUIREMENTS

2.5.1 Schedules and Milestones

2.5.2 R&M Related CDRLs

3 PROGRAM PLAN DETAILS

3.1 MONITORING AND CONTROL OF SUBCONTRACTORS/SUPPLIERS

3.1.1 Subcontractor/Supplier Program Controls

3.1.2 Selection and Surveillance

3.2 MAINTAINABILITY CRITERIA

3.2.1 Maintenance Concept and Maintenance Plan

3.2.2 Maintainability Requirements in Subcontractor Specifications

3.2.3 Maintainability Analysis

3.2.4 Maintainability Performance

3.2.5 Human Impact on Maintainability

3.3 SOFTWARE DOCUMENTATION

3.3.1 Problem Identification Reports (PIRs)

3.3.2 Impact of Software on Reliability

3.4 RELIABILITY GROWTH TESTING

3.4.1 Reliability Testing and Verification

3.4.2 Reliability Development Growth Testing (RDGT) Program

3.4.3 Production Reliability Acceptance Testing (PRAT)

3.4.4 Verification of Reliability

3.5 RM&A REVIEW INFORMATION

3.5.1 Quantitative Reliability and Maintainability Requirements

3.5.2 System Reliability

REV A

IV

3.5.3 System Maintainability

3.5.4 Availability

Figure 1. eCASS R&M Organization

Table 1 - Reliability Task Requirements Table 2 - eCASS R&M Related CDRLs Table 3 - RM&A Quantitative Requirements

REV A

1 INTRODUCTION

1.1 Scope

This Reliability and Maintainability (R&M) Program Plan describes the necessary tasks, responsibilities and controls that shall be implemented by the Contractor during eCASS Full Rate Production (FRP) 2.

The R&M Program Plan details approaches, processes, methodologies, products, organizational relationships, authority, and milestones relative to the management, analysis, surveillance, disclosure and testing necessary to achieve the program reliability and maintainability requirements.

This plan follows the general guidance of MIL-STD-785B, Task 101, and satisfies the requirements of the eCASS RFP 2 SOW. The primary function of the R&M effort is to document procedures, ensure both high operational readiness and availability, and to minimize life cycle cost. The plan shall address Commercial-Off-The-Shelf (COTS) and Non-Developmental Item (NDI) equipment to ensure that the eCASS specified R&M requirements are met or exceeded.

This Program Plan addresses the eCASS aspects in relation to:

Management

Schedule

Analytical Tasks

Control Tasks

Evaluation Tasks

1.2 Purpose

The purpose of this R&M Program Plan is as follows:

Define the R&M tasks to be accomplished

Define the organization for R&M and its interfaces to engineering, the program and other support organizations

Define the R&M management and control processes

Identify, describe and schedule the deliverable documentation

Describe Production Reliability Acceptance Testing (PRAT) for the program

Identify reporting requirements necessary for Integrated Logistic Support (ILS)

Describe maintenance data collection and failure reporting

The Reliability Program approach is one of defined tasks, planned progress sequences and progress milestones for results measurement. The program is intended to emphasize preventive as well as curative measures in the achievement of Reliability objectives. The preventive measures include:

REV A

1. Presence of on-site Subcontractor personnel as required for providing Reliability to ensure understanding and implementation of Reliability requirements;

2. Utilization of analysis as a Reliability assessment tool;

3. Support of comprehensive testing of the eCASS equipment predicated upon

Reliability and program requirements;

4. Performance of program reviews and orientation necessary to establish priorities for program emphasis;

5. Reliability engineering trade studies.

The curative aspects (early recognition of problem areas) of the planned Reliability program will include Monitor/Control of Subcontractors and Suppliers and rapid-response failure monitoring systems which emphasize effective reporting, corrective action and evaluation of the results of such action. The performance of corrective actions on reported failures, malfunctions, and deficiencies is predicated upon detailed analysis and reviews of designs, processes, corrective actions, and controls to determine the problem cause.

The program is structured to include the Reliability engineers in all decisions affecting system and software architecture, hardware, requirements definition and test requirements definition.

The Maintainability Program approach is to define the procedures used to ensure that the system complies with contractual maintainability requirements. Maintainability disciplines were applied during the design, and the primary effort was intended to ensure that quantitative and qualitative maintainability considerations were incorporated into the product.

The maintenance planning approach encompasses defining and establishing a maintenance concept, assessing and evaluating overall support capability, performing a support analysis to identify support resources, provisioning spares and repair parts, and developing controls for corrective action and modification. Maintenance planning provided critical processes early in the design phase and fed data to the logistics elements, which were dependent on the maintenance plan for fulfilling the requirements of their developmental analysis.

Factors in a maintenance plan include:

Defined actions to maintain system equipment in a defined state of operational readiness;

Developed maintenance procedures that include checkout, servicing, inspection, fault isolation, replacement and repair.

Conceptually, the output of maintenance planning identifies the level of maintenance at which each task will be performed, what support equipment will be required, task times and frequencies. Prior analysis provided input data to used identify spares, repair parts, consumables, technical data, facilities, manpower skills and training requirements.

REV A

1.3 Reference Documents

The following documents are referenced in this plan:

MILITARY STANDARDS

MIL-STD-785B

03 July 1986

Reliability Program for Systems and Equipment Development and Production

MIL–STD–471A

12 June 1995 Notice 3

MIL-STD-470B

30 May 1989

Maintainability Verification/Demonstration/Evaluation

Maintainability Program for Systems and Equipment

MIL–STD–1629

24 November 1980

Procedures For Performing a Failure Mode, Effects and Criticality Analysis

MILITARY HANDBOOKS

MIL–HDBK–472

12 January 1984

Notice 1

Maintainability Prediction

Mil-HDBK-470A, 12 June 1995

Designing and Developing Maintainable Products and Systems, Volume I and II

MIL-HDBK-781

14 July 1987

Reliability Test Methods, Plans and Environments for Engineering, Development, Qualification and Production

OTHER

SAE JA1000-1

Reliability Program Standard Implementation Guide

REV A

2 GENERAL

The vehicle for commitment to effective R&M engineering is the R&M Program Plan developed for the eCASS program. The R&M Program Plan emphasizes a comprehensive test, corrective action and demonstration program to identify and correct deficiencies as required. The eCASS R&M Program was implemented at the onset of development and Subcontractor/Vendor selection process. The COTS/NDI have existing maintenance support structures that are selected based on their compatibility with the specifications and maintenance concept of eCASS.

2.1 Organization

The R&M Program is the responsibility of the Lead Engineer and Program Manager who delegate that responsibility to the Logistics Group.

Within this group, the Lead R&M Engineer is responsible for conducting the eCASS R&M Program and will allocate requirements to the Hardware and Software IPTs, actively participate in said IPTs, assess the impacts to the system level R&M performance and provide the planning for the test phase as illustrated in Figure 1.

Active participation in the IPTs is essential to assess the impacts to the system level R&M performance and provide the planning for the test and support phases.

2.2 Functions

2.2.1 Reliability Tasks

Basic reliability tasks are developed using MIL-STD-785B as guidance. The tasks that are applicable to eCASS are selected and tailored specifically for the system and this phase of the program. The reliability tasks shall be performed in either a formal, informal or integrated manner, depending on the criticality of tasks, program requirements, benefits of performing the task and budget allowance. Tasks selected to be performed based on the contract modification are shown in Table 1.

eCASS Program Manager eCASS Lead Engineer

IPT Logistics Lead

R&M Lead Engineer

Figure 1. eCASS R&M Organization

REV A

Table 1 - Reliability Task Requirements

785B Task Number Description

101 Reliability Program Plan

102 Monitor/Control of Sub-contractors and Suppliers

105 Failure Review Board

304 Production Reliability Acceptance Test (PRAT)

Reliability Engineers utilize an R&M checklist to provide guidance to the design engineers to influence hardware design changes. eCASS equipment shall be evaluated throughout the change process, and detailed reliability assessments will be presented during periodic technical meetings or reviews. Reliability Engineers shall also monitor and evaluate system/subsystem level performance, including COTS/NDI, during the unit and system integration tests.

2.2.2 Maintainability Tasks

Maintainability engineers, as part of the eCASS program, shall perform all required maintainability tasks as required for this phase of the program using MIL-HDBK 472 (Maintainability Predictions) and MIL-HDBK 470A (Designing and Developing Maintainable Products and Systems) as guidance. Specific instructions, guidelines dissemination, and verification of compliance shall be accomplished and controlled in accordance with (IAW) the procedures and techniques outlined in this plan. No maintainability tasks were selected to be performed during FRP 2.

2.2.3 Support Tasks

Reliability and Maintainability engineers, as an integral part of the eCASS project team, will provide additional support tasks included herein to ensure program requirements are achieved.

2.2.3.1 Configuration Control Board (CCB)

Internal to the Contractor, the authority responsible for overall program baseline Configuration Management (CM) is the CCB. The CCB shall exercise control over the program Engineering Change Proposal (ECP) process. R&M engineering will support the CCB as required to ensure program objectives are not compromised by engineering changes.

The CCB monitors compliance to technical contract requirements in the processing and evaluation of the IPT initiated changes together with the review, assessment and response to directed changes. The board’s role and responsibilities are further defined in the eCASS Configuration Management Plan.

Another objective of the CCB is to formally release all drawing records and change documentation. Final approvals of changes are granted through this process only after concurrence is granted from all engineering disciplines. This allows logistics engineering to ensure that all reliability/maintainability/supportability requirements are met.

REV A

2.2.3.2 Peer Review Process

The peer review process is used to ensure all contractually required documents are technically accurate and reflective of requirements. This involves each engineering discipline reviewing and commenting on documentation pertinent to their respective area.

The comments are then discussed with the author and resolutions incorporated. Typical documentation reviewed by R&M Engineering via peer review includes design specifications, procurement specifications, hardware and software plans and procedures, and analysis and predictions.

2.2.3.3 Engineering IPT Meeting

IPT meetings for eCASS are led by the Project Engineer and are held bi-weekly to review the program status. In addition to technical discussions, programmatic issues are also discussed. If final disposition of issues cannot be obtained here, the issues are identified to the Integrated Management Team (IMT) for resolution. R&M Engineering is an active participant in these meetings.

2.2.3.4 Technical Reviews

The Contractor shall give the Government the opportunity to review the progress of the Contractor’s eCASS R&M Program. This information will be presented as requested during formal technical reviews. Logistics shall participate in all formal technical reviews scheduled for eCASS. The data provided shall include, as applicable at the time of the particular review:

R&M analyses and predictions

Potential problem areas and control measures

Effects of engineering decisions/changes and trade-offs on R, M, and BIT

Status of vendors R&M programs if applicable

Status of action items from prior reviews

2.2.3.5 Problem Identification Reports (PIRs)

The Contractor’s Logistics IPT, with assistance from the R&M engineering group, shall implement internal procedures for reporting problems identified during test that may affect R&M. These identified problems can be added to the Problem Identification Reporting (PIR) system based on Logistics IPT concurrence. The Problem Reporting (PR) system is used to ensure that the reporting of problems, which could result in a non-compliant system, are tracked, resolved and closed by individuals responsible for hardware, software, and documentation development. The system is a closed-loop system that tracks problems from discovery through various stages of resolution to final disposition of the corrective action. Upon review of a given PIR, an individual and/or department will be assigned to determine the resolution of the problem. The PR system shall be implemented at the beginning of the integration activities at test facilities and will continue through the life cycle of the program.

The action assignee is responsible for the generation of a corrective action report, or an equivalent report in the case of a Subcontractor, to identify the progress towards solving the problem or to update any existing information concerning the problem. The corrective

REV A

action is tested and verified and then forwarded to the PIR coordinator. The PIR coordinator will review the report for corrective action completeness, ensure that the required signatures are secured, and verify that the corrective action data is appropriately entered into the PR system.

As determined by the PIR coordinator, subcategories may be assigned to specific PIRs based on if the corrective action requires HW, SW or Test Program Set (TPS) support.

Closure of the PIR will require that all subcategories are sufficiently demonstrated as fixed to the appropriate Government representative. The Government representative will sign off on the PIR, indicating that the corrective action has been sufficiently demonstrated.

Multiple PIRs may be closed concurrently based on sufficient artifact data presented and on the Government representative validating the incorporation of specific corrective action that has fixed more than one PIR.

2.2.3.6 Data Collection Task Responsibilities

The Reliability Engineer is responsible for Failure Review Board (FRB) activities.

Reliability Engineering reviews the failure reports. Software failures generated during PRAT must be identified, documented and tracked. Failure reports will be written and used by Reliability Engineering.

Analysis is performed to determine whether any trends exist. Should a trend develop, Systems Engineering, Test Engineering, Manufacturing Engineering, Quality Engineering, and the Government will be notified, and a team will be assembled to resolve the problem by identifying the trend and implementing and monitoring corrective action.

2.2.3.7 Quality Engineering Interface

The Quality Engineer has prime responsibility for assuring the a failure reporting program is implemented for supplier attributable problems. This process includes:

Processing reports of discrepant material

Interfacing with Receiving and Operations for component/unit receiving inspection

Working with System Integration to ensure test acceptance

Monitoring factory in-process module, unit, and subassembly testing

Monitoring overall program performance

Data associated with supplier asset problems will be documented by the R&M Engineer.

Quality Assusrance (QA) and R&M will work together to ensure the proper level of failure analysis and the corrective actions performed by the suppliers align with the Contractor’s standard process. Quality Engineering will generate a supplier report, if required, due to a reliability engineering identified trend failure or if further data is required to satisfy eCASS Contractor’s Failure Review Board (FRB) program needs.

2.3 Duties

The goal of this R&M Program is to minimize R&M risks. eCASS System Reliability and Maintainability requirements have been established and integrated into system change/ modification requirements. R&M risks are identified throughout the program and

REV A

shall be controlled by proven R&M analyses, prediction tools, data collection and corrective action system as described in the following sections.

2.3.1 Reliability and Maintainability Engineering

The R&M Engineer has the prime responsibility for the Failure Review Board (FRB) activities during the system integration and test period. The tasks associated with eCASS are:

Monitoring and collecting all pertinent test data relating to each failure

Identifying failure trends

Identifying problem areas that necessitate further investigation

Initiating and coordinating failure analysis activity

Instituting and coordinating the necessary corrective actions

Ensuring that eCASS can achieve the R&M specification requirements

To ensure accurate and timely failure data recording and processing of failures, Reliability specialists shall monitor all areas of test for which R&M Engineering is responsible. These specialists shall be responsible for performing program demonstrations, initiating failure reports, recommending detailed unit and part failure analysis requests, identifying failure trends and evaluating corrective action follow up.

2.3.2 Technical Reviews

The purpose of technical reviews is to work through the technical issues. Participation in technical reviews provides the R&M Engineers with a vehicle to ensure that the specifications for quantitative and qualitative requirements are met. Many of these requirements are imposed at the equipment level, and engineering judgments are made in a cost-effective manner by considering hardware cost, through-life cost, operational availability, and achievement of specification. If the R&M requirements are not being met, changes/modification recommendations are made to the Project Engineer and technical IPT for discussion and resolution as appropriate. If R&M issues or risks are not completely resolved through a technical review, they will be elevated to program management for resolution.

2.4 Responsibilities

The mission of R&M Engineering is to ensure that delivered hardware achieves the highest practical standards of Reliability and Maintainability and meets all contractually specified requirements. This will be accomplished by R&M Engineering. Specific responsibilities at this point in the program include:

Prepare, update, implement, and manage the R&M Program Plan.

Ensure all supplier and Subcontractor supplied products achieve allocated R&M requirements.

Actively participate in the on-going change efforts.

Perform quantitative and qualitative analyses IAW the updated eCASS Statement of Work (SOW) for configurations.

REV A

Assess impacts of resultant product changes/modifications on system-level goals and reassign allocations when needed.

Attend program reviews and technical reviews to assure compliance with the R&M requirements.

Perform and document reliability, maintainability and supportability analyses and trade-offs to determine alternatives as required.

Collect and analyze failure data during functional, demonstrational, and operational tests.

Be responsible for the failure analysis coordination, remedial action, and corrective action activities associated with the Contractor FRB.

Assisting in COTS/NDI equipment selection and sub-contractor/vendor specifications development if required.

Obtain data on maintenance frequency, and ensure effective resolution of any associated problems in achieving program requirements.

Coordinate the safety aspects of maintenance with appropriate engineering activity.

2.4.1 Reliability and Maintainability Program Integration

Reliability will interact with Logistics Engineering to ensure the reliability performance of each subsystem is accurately recorded IAW contract requirements.

Reliability Engineering will support, as required, the pertinent reliability aspects of Human Engineering, Safety Engineering, Quality, Configuration Management, Systems Engineering, Mechanical Engineering and Software Engineering. These groups as required will in turn support Reliability Engineering. Support shall include but not be limited to:

Provide failure rate data to Systems Engineering in support of system analysis and trade studies.

Provide failure rate data to support maintainability predictions and fault monitoring analyses; also, obtain maintenance concepts and ground-rules for reliability analyses and predictions to ensure compatibility as required for the contract.

Hardware Engineering provides hardware estimates and bills of material for each hardware function and part stress data, if necessary. For COTS/NDI data, they provide vendor data and estimates.

Quality Engineering will be the key interface in the collection and evaluation of supplier failure data.

Systems Integration and Test Engineering to provide assistance in Reliability Demonstration during PAT/PRAT.

REV A

2.4.2 Maintainability Interface Compatibility

The Maintainability Program requires interface with other disciplines to ensure that the integration/installation of eCASS subsystems meet fault detection, fault diagnostics and repair objectives at the intermediate and depot levels of application.

The maintainability interface shall:

Obtain failure rate data to support maintainability assessments.

Provide maintainability data to support availability analyses and testability analyses.

Review the maintenance concept with customer Logistics Engineers for maintainability analyses to ensure compatibility.

Work with Systems and Hardware Engineering to obtain the system baselines, hardware drawings, and Bill Of Materials (BOM) required for analysis.

Work with the Logistic Cost Engineering team to perform through-life cost analysis, as required.

Work with Systems Integration and Test for maintainability demonstration coordination assistance.

2.4.3 Failure Review Board (FRB)

The FRB provides a mechanism to resolve failures that occur during all system integration, test, and Production Reliability Acceptance Test (PRAT). [Tool] will provide tracking, control, and traceability of reported failures from occurrence to closeout for production failures. Field failures will be tracked in [tool] which contains failure reporting, analysis, and corrective action capabilities.

There are three basic phases of FRB process:

Collecting failure data – this has been established and is currently planned to continue through FRP 2

Combining test data with prediction data

Using field data

The first phase of the FRB is accomplished by establishing a coordinated effort between the test group, Reliability Engineering and the customer R&M IPT.

The second phase is the combination of test data with prediction data for better assessment of the R&M parameters of the system. The collected test data captured by the FRB database will be used to update the predicted station performance.

The final phase of the FRB is the collection of in-service field data as a means for identifying reliability and maintainability parameters and errors and for initiating corrective actions while the system is in Navy use.

As indicated, R&M Engineers use the failure data collected as a basis for updating the prediction analysis and for scoring review activities with the customer as part of the Contractor Failure Review Board (FRB). If unresolved problems are disclosed as a result of the FRB activities, corrective action can be instituted. Corrective action encompasses multiple responses by R&M personnel. Direct communication with NDI equipment

REV A

vendors and engineering disciplines is utilized extensively for problem resolution.

Corrective action items may be addressed in program status meetings as requested.

The R&M Engineering group will work closely with equipment vendors to resolve R&M problems. Action items are assigned to each engineering discipline with due dates specified for required resolution. The resulting assessment data will include failure analysis data and shall be entered into FRB database and also into the analysis process.

Data collection commences from unit tests through system integration tests and continues into FRP 2 and the field.

2.4.3.1 Failure Data Collection

Reliability Engineering will monitor the test logs and obtain the technical data for any incidents that occur. When an apparent malfunction occurs, test technicians and Systems Engineering isolate the cause of the malfunction, make proper entries into the logbook, and initiate an Problem Identification Report (PIR) so that the equipment can be repaired and/or returned to the operational state. An investigation will be performed by the Systems Engineering team to determine the root cause of the issue and to determine if the anomaly should be attributed to hardware, TPS code, or software.

In the case of hardware replacement, a [mechanism] is written documenting the discrepancy. Detailed hardware data (type, part number, and serial number), failure data, and replacement and repair information will be entered into the [mechanism] form by Engineering/Test and Repair personnel. The failure data will be reviewed by the R&M Engineer to define significant failure trends and to identify reliability problems requiring additional investigation.

Defective COTS/NDI subassemblies will be forwarded to the supplier through the normal return process for evaluation and identification of the mechanism of the failure. The OEM supplier will provide the repair data to Reliability Engineering and develop steps necessary to correct the subassembly defect. Non–repairable subassemblies shall be replaced, and failure data shall be monitored for trends. When a trend is determined, Reliability Engineering will head a team to identify root cause and ensure corrective action is implemented.

In the case of a TPS or software issue, a [tool] issue will be generated in a standard tracking database. The [tool] record will be used to investigate and document the required code changes. Proposed code changes are reviewed and approved by the software CCB prior to code check-in and test demonstration. [Tool] records will be reviewed by the R&M Engineering team to evaluate root cause analysis, search for failure trends and to identify reliability problems requiring additional investigation during the PRAT phase or as required. PRAT records will be added to FRB database, as appropriate, indicating station software or hardware issues, which have resulted in relevant failures.

2.4.3.2 Failure Reporting

Failure reports will be generated and provided to the customer IAW contract requirements.

For eCASS, the failure data is provided within the FRB database.

2.4.3.3 Failure Verification/Failure Analysis/Corrective Action

Failure verification procedures are in place to ensure timely resolution of problems at the station level and confirm failed assets are returned for repair. Technicians and Systems

REV A

Engineering will use various techniques to verify defective assets, such as testing on the bench or in other stations.

The supplier or vendor will be consulted when a failure trend of two or more assets with the same failure mode has been identified. The supplier will be informed that a formal failure analysis is required for trend failures. The supplier will then perform detailed failure analysis as requested by Engineering. The R&M Engineering group will provide assistance in failure analysis when the supplier does not have the capabilities. The intent of the failure analysis activity is to identify and correct any hardware reliability deficiencies as early as possible. The failure verification and analysis are coordinated by Reliability Engineering, with the assistance of on-site Hardware, Mechanical, Manufacturing, and Materials Engineering.

Corrective action recommendations will be offered, and disposition will be determined as necessary to correct reliability deficiencies identified in eCASS program equipment. The reliability of hardware receiving corrective action will be monitored on a continuing basis to ascertain the effectiveness of the corrective action and to assure that the highest possible reliability is attained.

2.4.3.4 Elapsed Time Indicators (ETIs)

Each eCASS Station shall include an ETI used as a means for logging the equipment operating times. The ETI will be used during formal test events to identify time to failure and provide a summary of total hours in order to determine Mean Time Between Failure (MTBF) values.

2.5 Reporting Requirements

2.5.1 Schedules and Milestones

The Lead R&M Engineer shall ensure that each R&M task is scheduled and that each resource is loaded to ensure program objectives are achieved. Tasks performed for eCASS are contained within the Integrated Master Schedule (IMS) to support the Reliability and Maintainability CDRL deliveries and PRAT. Please reference the eCASS program Integrated Master Schedule (IMS) to view schedules and milestones of all R&M tasks.

2.5.2 R&M Related CDRLs

CDRL deliverables are submitted to the Government to provide a permanent record of accomplishments under the R&M Program. These deliverables shall address items described in the contract. The R&M related contractual deliverable items for eCASS FRP 2 will include the following:

Table 2 - eCASS R&M Related CDRLs

CDRL Name

A007 Problem Identification Report (PIR)

A008 Engineering Change Proposal (ECP)

A00A Reliability and Maintainability (R&M) Program Plan

REV A

3 PROGRAM PLAN DETAILS

3.1 Monitoring and Control of Subcontractors/Suppliers

3.1.1 Subcontractor/Supplier Program Controls

Subcontractors/Suppliers are contacted frequently to ensure continued understanding of the R&M requirements. These requirements are included in procurement specifications imposed on vendors to ensure continued procurements of reliable units, subassemblies, components, and materials. Engineering will control these documents using vendor item or specification control drawings. The documents shall include vendor controls on processes, materials, packaging, screening, lot testing, and electrical parameters, as is required for meeting the program’s R&M requirements.

R&M Engineering will also review all vendors’ predictions, failure data, and repair data to ensure accuracy of approach and to eliminate causes of unreliability.

3.1.2 Selection and Surveillance

The IPT will utilize a rigorous vendor selection process to meet the overall R&M requirements. The major focus is to ensure vendor COTS and commercial NDI equipment product reliability and quality can be substantiated and can perform as expected within the station. Evaluations of estimated versus real time performance will be presented during program and technical reviews. Analysis will be performed to reflect real time performance versus predicted commercial NDI failure rates.

Engineering, Quality, Purchasing, Supportability & Specialty Engineering, and Quality Assurance continuously evaluate Subcontractors/Suppliers to determine if performance is as expected. Surveillance of vendors’ R&M procedures, quality audits, technical reviews, or other improvement efforts may be performed to ensure vendor compatibility with overall program requirements.

3.2 Maintainability Criteria

The eCASS Maintainability Program is intended to improve operational readiness, reduce maintenance manpower needs and through-life costs, and to provide data essential for management to assess manufacturing, integration and testing within the eCASS program.

Maintainability Engineering influenced the eCASS NDI physical design selection/modification to ensure an optimum balance among maintainability, cost, and program effectiveness.

Checklists of attributes and criteria for evaluation of maintainability provide guidance to aid in the selection process for the NDI equipment. The application of criteria ensures compliance with all eCASS maintainability criteria. Typical areas evaluated include:

Accessibility/work space

Interchangeability

Fault detection and isolation techniques and capabilities

Special tools and support equipment

Maintainer skills and requirements

REV A

Testability and test points

Training requirements

Transportability and handling

Supportability requirements

Compatibility of NDI equipment and updates

Application of the criteria provided by the Logistics IPT ensures compliance with maintainability requirements and supports the optimization of eCASS availability, support resources, and reduced maintenance burden. Analysis criteria established by Maintainability Engineers is available to ensure that maintenance planning tasks are compatible with maintainability requirements.

3.2.1 Maintenance Concept and Maintenance Plan

The maintenance concept has been defined as a two-level support for intermediate and depot level maintenance. This decision was made based on maximizing maintainability and making it possible to minimize reliance on high skill levels, extensive documentation, and training. The use of commercial and NDI equipment also helps clarify levels based on the benefit of established vendor maintenance concepts, procedures, and supportability.

The eCASS maintenance concept emphasizes use of automatic FD/FI on electronic equipment for quick removal and replacement at Intermediate and Depot levels and for supplementation of manual troubleshooting procedures for certain low frequency tasks, such as cable and backplane faults.

3.2.2 Maintainability Requirements in Subcontractor Specifications

Maintainability Engineering flow-downs impose the same contractual requirements as those imposed on the eCASS IPT and review all non-COTS Subcontractor’s preliminary design packages accordingly. Request for modifications for items adversely affecting maintainability shall be made through the Project Office. The Procurement Specifications and SOW provide the definitions of the requirements to ensure compliance to system specifications and operational requirements.

3.2.3 Maintainability Analysis

R&M Engineers review all COTS equipment selected to assure maintenance requirements are achieved and that considerations are given to diagnostic and repair levels. A Maintainability Analysis conducted throughout SDD and LRIP programs has confirmed that maintainability requirements are being achieved. The primary intent of the Maintainability Analysis is to translate maintenance concepts, requirements, and constraints into detailed quantitative and qualitative maintainability requirements. A direct result of the analysis is the identification of potential maintenance and support problems early. The analysis was performed at all levels within the design. The results of the Maintainability Analysis are implemented through internal technical reviews, written reports to the Project Office, asset selection, and equipment specifications.

3.2.4 Maintainability Performance

An effective maintainability concept requires high emphasis on features, such as Built-In- Test (BIT) and fault isolation, and acknowledges the criticality of these features to the effectiveness of system testability and maintainability. The eCASS BIT goals are the

REV A

attainment of the highest fault coverage detection possible and the isolation to the minimum number of Line Replaceable Units (LRUs). The BIT requirements for GFE adhere to its original specification and were verified by testing, data, or analysis of data.

BIT assesses the functional system performance and identifies failed LRUs to determine operational readiness.

eCASS maintainability performance is addressed in terms of maintenance parameters, both quantitative and qualitative, i.e., levels, functions accomplished at each level of maintenance, support policies, assessment factors (MTTR, Maximum Corrective Maintenance Time (Mmax), etc.), and logistic support requirements. Support data, including maintainability predictions and repair data, were provided for each subsystem, excluding GFE except for those items reused and retained using newly designed eCASS retention features.

For commercial equipment that was not developed using military specifications, the maintainability and testability characteristics of the equipment are addressed utilizing support data, such as actual repair data, test data, and maintainability predictions.

3.2.5 Human Impact on Maintainability

The man machine interface was of primary concern during the development phase of the eCASS program. The Maintainability Engineer’s primary concern in this area was the ability of the maintenance personnel to gain access to the failed station assets and to be able to fault isolate to the proper level in order to facilitate repairs.

3.3 Software Documentation

3.3.1 Problem Identification Reports (PIRs)

CDRL A007, Problem Identifcation Reports (PIRs), are used to document all proposed changes or discrepancies in the baseline software and associated baseline documentation and are acceptable for change control of both the informal and formal baselines.

Nonconformance detected in the products under configuration control are documented and tracked using the PPR process as defined in the CM plan. This closed-loop system uses identification, analysis, disposition, and verification steps to implement corrective actions.

Firmware updated on the station will be recorded similar to software; however, assets returned to the OEM for firmware updates will be recorded as hardware issues. The analysis of operational software and firmware indicates where more attention to reliability/quality is required. It also provides indicators as to where improved techniques should be instituted throughout the project.

3.3.2 Impact of Software on Reliability

Reliability Engineering is incorporating software issues into the FRB database that occurred during PRAT and is performing assessments to indicate reliability growth. The results of these analyses will be incorporated into the updated FRB database.

3.4 Reliability Growth Testing

All production acceptance test activities at the subsystem and system level, including Reliability Growth Testing, will be monitored to ensure that all eCASS equipment shall

REV A

meet specified reliability performance requirements. Failure data shall be collected and analyzed for trends. Subcontractors will provide failure analyses and corrective actions of their units or subassemblies wherever necessary. The R&M team shall continue to conduct RGT until the stations enter Production Acceptance Testing (PAT) / Production Reliability Acceptance Testing (PRAT) in order to demonstrate that the system reliability performance meets the requirements with the specified level of confidence.

3.4.1 Reliability Testing and Verification

The Reliability testing program planned for eCASS collects data from all types of testing.

From initial board testing and through acceptance testing, failure data is collected and analyzed for trends in components, vendors, or design problems. Critical items (components, assemblies) are also tracked and analyzed for potential problems. This is an ongoing program, not one that relies on a certain time period to test for problems to occur and then decide what to do, but one that is continually working to identify and correct problems as early in the build process as possible. The process provides assurance that all significant problems have been detected and resolved prior to platform test and production release.

Verification of achieving reliability requirements is defined as passing predetermined levels of Mean Time Between Failure for each configuration. Analysis is performed through predictions and confirmed through demonstration testing with respect to schedule, cost, and system architectural constraints. The approach applies the Test-Analyze-And-Fix (TAAF) process to the entire test program for eCASS, using available test data to monitor reliability status.

3.4.2 Reliability Development Growth Testing (RDGT) Program

RDGT must correct failures that reduce operational effectiveness and failures that drive maintenance and logistic support costs. RDGT is a planned, pre-acceptance test process that uses a TAAF approach, in which stations are tested under actual, simulated, or accelerated environments to disclose deficiencies and defects. This testing has resulted in early incorporation of corrective actions and verification of their effectiveness, thereby promoting reliability growth and increasing station reliability. The multiple sequences of testing, analyzing failures, and implementing corrective actions documented through FRBs has provided sufficient data for early establishment and continued monitoring of system reliability status.

TAAF is implicit to the FRB program; the basic motivation is to find and fix hardware and software problems using data documented by the FRB during testing. Corrective action is taken where failure analysis and statistical failure data identify trends. Unique to the eCASS TAAF program is the requirement to evaluate NDI/COTS equipment items.

Problems identified with NDI items may be resolved by working with the OEM to incorporate minor changes to the NDI. COTS products will require participation of the product vendor or alternative source selection to resolve COTS issues. The accumulation of data for reliability growth monitoring as documented in FRB database is being provided to the customer.

3.4.3 Production Reliability Acceptance Testing (PRAT)

A PRAT Plan was prepared,outlining the scope and requirements to be satisfied by these tests, and was submitted as part of the PAT/PRAT test procedure. PRAT will be performed IAW MIL-HDBK-781 test plan XVIII-D and will consist of a 100-hour failure-

REV A

free burn-in test, consisting one run of foreground SMAT TPSs and any supplemental tests followed by repeating background SMAT TPSs until 100 hours has elapsed.

Detailed test procedures are utilized from eCASS LRIP. Specific acceptance criteria including specified confidence levels or decision risks ("Upper Test MTBF," "Lower Test MTBF," etc.) are defined using the statistical test plan approved by the customer.

3.4.4 Verification of Reliability

The reliability of eCASS will be determined from the test data collected compared against the predicted and required specified reliability, based upon problem corrections. The data collection methods will provide the necessary information for assessing reliability validation during the testing periods.

The reliability growth rate is monitored by comparing achieved reliability progress with the planned reliability rate using the mathematical model selected for this purpose. An achieved reliability growth rate that compares favorably to the specified growth rate testifies to the effectiveness of the TAAF Program. If the growth is significantly less than planned or the testing is taking too long, a careful analysis will be made to determine the reason for poor performance.

The results of the analysis will be presented to the Failure Review Board for review and possible modification of the TAAF methodology. The projected, actual, and potential reliability growth curves and other data showing progress toward attainment of the reliability requirements will be presented at scheduled program review meetings.

3.5 RM&A Review Information

Reliability, maintainability, and availability requirements are incorporated through R&M engineering’s participation in the system manufacturing, integration and testing processes.

The system level tasks begin with analysis of the specification requirements and proceed down to the Hardware Configuration Item (HWCI), Software Configuration Item (SWCI), or Unit level. R&M engineering is an integral part of the technical review process applied to all HWCI equipment and SWCI products.

The R&M program has been planned and scheduled to permit program management and the Government to review the progress of tasks and evaluate the achievement of RM&A requirements at appropriate stages of the program through formal and informal reviews.

The information in this section and subsections are only applicable to FRP 2.

3.5.1 Quantitative Reliability and Maintainability Requirements

The quantitative RM&A requirements for eCASS, as indicated in the specification system quality factors, are defined in terms of station reliability (expressed as MTBF), unscheduled maintenance or Mean Time To Repair (expressed as MTTR), station availability (expressed as Ao), and scheduled maintenance. Scheduled maintenance support requirements are defined by hours per event and per year. Additional requirements are defined under station self-test for Mean Time Between False Alarm (expressed as MTBFA). Specified eCASS program RM&A metric values for each configuration are indicated in Table 3.

REV A

Table 3 - RM&A Quantitative Requirements

Requirement Metric (Excluding GFE) Mean Time Between Failure - MTBF (Hours) >584 hours

Mean Time To Repair - MTTR unscheduled (Hours) <0.98 hours

Maximum Time To Repair – Mmax <2.7 hours

Operational Availability – Ao >94%

Scheduled Maintenance (per event) <8 hours

Scheduled Maintenance (per year) <24 hours

Mean Time Between False Alarm - MTBFA >8700 hours

3.5.2 System Reliability

The system, for the purpose of reliability, shall be comprised of four station configurations that will support specific mission requirements. The overall design for the eCASS Station consists of a four-rack eCASS Station that includes common subsystems used in all eCASS Missions. The defined missions are to provide capability to execute the eCASS Hybrid, Radio Frequency (RF), High Power (HP) and Electro-Optical (EO) missions. The EO, HP, RF and Hybrid configured stations shall have a Mean-Time-Between-Failure (MTBF) of no less than 584 hours each excluding GFE (i.e. EO Console, HP Console, etc.). Hardware and Software changed/modified under this contract will meet the reliability, maintainability, and supportability requirements.

3.5.2.1 Reliability Analysis

Reliability modeling, allocation, and prediction activity provide the basis for evaluation of the system and subsystems against their requirements, for identification of problems for corrective actions, for identification of areas for reliability improvements, for allocation of requirements to lower levels for development of supplier requirements. The development of reliability mathematical models, reliability predictions, and FMECA is not intrinsically any different because of the use of COTS/NDI. The use of COTS/NDI hardware requires careful review and analysis of vendor data in the prediction of failure rates.

eCASS hardware reliability requirements have been addressed in the reliability predictions. The predictions included analyses for individual subsystems and/or equipment and system reliability. The FMECA was performed against equipment items to identify critical functions and to establish mitigations required to reduce failure risk to acceptable levels.

3.5.2.2 Failure Modes, Effects and Criticality Analysis (FMECA) / Fault Tree Analysis

A FMECA was performed at the subsystem and/or Line Replaceable Unit (LRU) level to identify and evaluate potential design weaknesses and the effects of failures on the system performance. At the LRU levels, the effect of the incidents or faults shall be categorized by their impact on the functions performed by eCASS.

REV A

The FMECA was performed IAW the requirements of MIL-STD-1629 and evaluated the impact of failure modes at the system and subsystem levels. The severity categories are used as follows:

Category I (Catastrophic)—A failure that may cause death or mission system loss.

Category II (Critical)—A failure that may cause severe injury, major property damage, or major system damage that will result in mission loss.

Category III (Marginal)—A failure mode that may cause minor injury, minor property damage, or minor system damage that…