FinalMSEReviewGuide.pdf

FOR REFERENCE & GUIDANCE PURPOSES ONLY

Single Manager for Conventional Ammunition

(SMCA)

Review Guide

Measurement System Evaluation

(MSE) Requirements http://www.pica.army.mil/PicatinnyPublic/organizations/ardec/orgchart/quality.html

30 June 2014 http://www.pica.army.mil/PicatinnyPublic/organizations/ardec/orgchart/quality.html

MSE Review Guide

Table of Contents

Page No.

Background 3 – 4 Introduction 5 Paragraph (a) – Definitions 6 – 7 Paragraph (b) – Scope 8 – 10 Paragraph (c) – AIE 11 – 14 Paragraph (d) - AIE Designs & Government

Furnished Gages 15 – 16 Paragraph (e) - AIE Package Submittals 17 – 18 Paragraph (f) - Characteristics for Inspection 19 Paragraph (g) - Automated Acceptance Inspection

Equipment 20 – 31 Paragraph (h) - Measurement System Analysis (MSA) 32 – 33 Paragraph (i) - Robust AIE System 34 Paragraph (j) - AIE Calibration and Verification 35 – 36 Paragraph (k) - Non-Destructive Testing (NDT) 37 – 43 Paragraph (l) - Contractor Alternate Inspection

Method(s), Modifications and/or Relocation of AIE (Non-Automated) After Government Approval 44 – 50

Paragraph (m) - Responsibility for AIE Package Submittal 51 – 52

Paragraph (n) - Government’s Right to Disapprove

AIE 53 – 54

Paragraph (o) - Navy Furnished Gages 55 – 57 References 58 – 59 Attachment 1 – MSE Contract Data Requirements

List (CDRL) Example (DD Form 1423) 60 – 62 Attachment 2 – MSA Contract Data Requirements

List (CDRL) Example (DD Form 1423) 63 – 64 Attachment 3 – Data Item Description (DID) 65 – 76 Figure 1 – Minimum Number of Hits for CMM Inspection 73 Figure 2 – AIE Process Flow Chart 77

An MSE Review Guide is available to assist Contractors on the ARDEC Public Website in the application of the MSE Clause. The MSE Review Guide's content, in its entirety, is provided solely for REFERENCE and GUIDANCE PURPOSES ONLY - it is not, nor is it intended to be, contractually binding. Accordingly, the information contained within the MSE Review Guide is expressly not, nor shall it be construed to be, incorporated either directly or by reference into the terms of the MSE Clause itself nor into the terms and conditions of any underlying contract which contains the MSE Clause.

Background The Measurement System Evaluation (MSE) clause was developed by the Single Manager for Conventional Ammunition (SMCA) and Military Services as a tool for use in ammunition acquisition and, when applicable, in conjunction with other supplier quality requirements.

MSE was established to replace the three current Acceptance Inspection Equipment (AIE) clauses:

1. Army/Joint Munitions Command

2. Navy/Air Force/Joint Munitions Command

3. ARDEC

The MSE clause was designed with continuous improvement in mind to complement the four pillars of supplier quality which are:

• Prevention – Process Capability, Control & Improvement

(PCCI)

• Critical Characteristic Control (CCC) – Detection, prevention & continuous improvement for safety critical characteristics

• Quality Management System (QMS) – Framework for continuous improvement

• Detection – Measurement System Evaluation (MSE)

The MSE clause does not create any new AIE requirements but serves to highlight and emphasize requirements that have been in existence in Technical Data Packages and contracts all along.

AIE requirements have been spread out across a variety of different documents such as MIL-A-70625A (Automated Acceptance Inspection Equipment Design, Testing and Approval of, 21 April

1989), MIL-A-48078A (Ammunition Standard Quality Assurance Provisions, General Specifications For, dated 16 December 1988), and the Critical Characteristics Clause. The MSE clause, along with its corresponding MSE Contract Data Requirements List (CDRL), Measurement System Analysis (MSA) CDRL, MSE Data Item Description (DID), and MSE Review Guide consolidates all AIE requirements into one location.

Other areas of MSE, such as calibration systems, MSA, and use of process control systems for acceptance of product are also addressed. The MSE clause and associated information seek to utilize best industry practices for:

1. Prove-out methodologies/requirements pertaining to AIE, Automated Acceptance Inspection Equipment (AAIE), and Process Control Systems.

2. DID, CDRLs, and review guide that provide clarity to contractors on AIE submissions and the Government on how to review those submissions, within the associated time lines.

3. Applicability of MSE requirements to Performance Based or Commercial Off-the-Shelf (COTS) acquisition.

4. Appropriate references to latest commercial industry standards.

5. Storage, handling, and environmental conditions associated with inspection systems.

6. Government Furnished Equipment (GFE) requirements from Navy/Air Force/JMC clause.

Requirements that adequately address modern measurement systems for inspection (and process control when used for acceptance), and a robust methodology to evaluate these systems is projected to enhance the reliability for acceptance of conforming product.

Consequently, there will a smoother review and approval process for AIE submissions due to improved synchronization between Government and contractor expectations. The guide will assist personnel responsible for interpreting and executing the requirements of the contract.

To reiterate, MSE requirements are intended to be uniform, integrated requirements for the SMCA or other procuring agencies to aid suppliers in accomplishing their AIE work while encouraging them to utilize commercial best practices. The MSE clause supports and reinforces the expectations of MIL-A-70625A and MIL-STD-1916 (DOD Preferred Method For Acceptance Of Product, paragraph 4.4, 1 April 1996).

Introduction The purpose of this Review Guide is to assist personnel involved in the review, design, or selection of AIE utilized for inspection of characteristics identified in Government Technical Data Packages (TDPs).

The MSE Review Guide is meant to be an aid for:

- Applying the MSE clause.

- Developing AIE submissions.

- Reviewing AIE submissions.

The document is formatted in a fashion where a clause paragraph is followed by a discussion of what is intended by the paragraph and what should be contained in an AIE submission to address that particular clause requirement. Paragraphs of the clause are included in the review guide but are not substitutes for the actual requirements included in a solicitation, purchase order or contract.

There are a variety of approaches/methods/tools that can be utilized to comply with requirements. The goal in developing the review guide is to provide guidance of the MSE requirements.

The Integrated Product Team (IPT) will consist of Government, Contractor and sub-Contractor representatives, as appropriate.

It is recommended that the Contractor participate in Government partnering through IPTs during the analysis. This will facilitate review of the AIE submissions and allow for open discussion of the manufacturing, inspection and material handling processes.

Contracts & Statement of Requirements – COCOs, GOCOs & GOGOs

The term Contractor is used throughout this document to denote the entity with a Government contract or Government Owned, Government Operated (GOGO) activity responsible for preparation and submission of AIE design for approval. The normal method of transmission of MSE requirements for Contractor Owned, Contractor Operated (COCO) and Government Owned, Contractor Operated (GOCO) facilities is via a contractual instrument. The method of transmission of MSE requirements at a GOGO is through a Statement of Requirements (SOR) via a Production Work Directive.

Paragraph (a) Definitions

Discussion:

Paragraph “(a) Definitions” of the clause is used to define commonly used terms in MSE.

(a) Definitions. This paragraph defines specific terms utilized throughout the rest of the clause and in the accompanying Contract Data Requirements List (CDRL) and Data Item Description (DID) (DI-QCIC-81960). This aids in clarifying the MSE requirements to Government and contractor personnel.

(1) Acceptance Inspection Equipment (AIE). All equipment (includes AAIE defined below), special and standard, including dimensional gages, measuring equipment, test fixtures, electronic and physical test equipment, and other test equipment used for examination and test of a product to determine conformance to the Technical Data Package (TDP) which may include drawings and specifications (e.g., Detail, Performance, Weapon specifications, and QAPs).

(2) Automated Acceptance Inspection Equipment (AAIE). AIE in which the inspection and acceptance determination of the product is performed, in whole or in part, in an automatic manner.

(3) Contractor Inspection Equipment. Government-approved equipment utilized by the contractor to perform examination and tests to assure conformance to contract requirements.

(4) Commercial Inspection Equipment. Industry-developed inspection equipment of universal application, without limitations to a specific part or item, which is advertised or cataloged as available to the trade or to the public on an unrestricted basis at an established price. Examples follow:

(i) Standard Test Equipment. Multiusage equipment that is specific to a function rather than to an item. It includes such items as hardness testers, tensile strength testers, meters, weighing devices, standard gear testers, ohmmeters, voltmeters, and oscilloscopes.

(ii) Standard Measuring Equipment (SME). Multipurpose equipment and standards used for performing measurements. It includes such items as micrometers, rulers, tapes, height gages, and protractors, etc. Standards include visual inspection equipment such as scratch and dig standards, surface finish comparator, color standards (FED-STD-595), etc.

(5) Nondestructive Testing. The development and application of technical methods to examine materials or components in ways that do not impair future usefulness and serviceability in order to detect, locate, measure and evaluate flaws; to assess integrity, properties and composition; and to measure geometrical characteristics. NDT includes Radiography/Radioscopic, Ultrasonic, Eddy Current, Magnetic Particle, and Liquid Penetrant.

(6) Measurement System Analysis (MSA). Per ASTM E2782 (Standard Guide for MSA), paragraph 3.1.7, MSA is any of a number of specialized methods useful for studying a measurement system and its properties.

During MSE Industry Day of 10-11 December 2013, the Supplier Quality Initiative (SQI) Working Group (WG) was tasked to expand on the definition of AAIE. The definition of AAIE was extracted from MIL-A-70625A and is repeated here for convenience:

Automated Acceptance Inspection Equipment (AAIE). AIE in which the inspection and acceptance determination of the product is performed, in whole or in part, in an automatic manner.

The concern focused on what is meant by “in whole” or “in part” in the definition. MIL-A-70625A does not elaborate on what is meant by “in whole” or “in part.” As the definition is written, it implies that the “acceptance determination” of the product can be performed completely (in whole) or partially (in part) by the AAIE. In fact, the definition could be refined to mean, “AIE in which the inspection and acceptance determination of the product is performed in an automatic manner.” Hence, the definition of AAIE as presented in MIL-A-70625 will be updated accordingly in a future continuous improvement effort.

After speaking to the original author of MIL-A-70625, the words “in whole” or “in part” dealt with the amount of automation of the accompanying Material Handling System (MHS) to the AAIE.

That is, the MHS can be completely (in whole) or partially (in part) automated. “In whole” would mean that the MHS has the capability to identify and segregate “good” from “bad” product while “in part” means the MHS would involve some kind of human intervention. However, the “inspection and acceptance determination of the product” by the AAIE is independent of the

MHS.

Paragraph (b) Scope

Each sentence of paragraph “(b) Scope” of the clause is discussed and elaborated on below.

First sentence: This clause establishes requirements for design, supply, performance, and maintenance of AIE used for product inspection and acceptance.

• The MSE clause outlines the requirements that need to be satisfied when an AIE is designed for inspection of desired TDP characteristics.

• “Supply” of AIE refers to the act of the Government providing the AIE designs or actual gages to the contractor for use, or the contractor providing its AIE designs to the Government for review and approval prior to fabricating the corresponding inspection equipment. Procedures for development of AIE documentation to be submitted to the Government for review and approval shall be in accordance with CDRL (DI-QCIC-81960). Any AIE hardware/software that is produced prior to Government approval of the submitted gage package will be at the contractor’s own risk.

• “Performance” of AIE refers to the inspection equipment functioning as required (accurate, reliable, and repeatable) where there are no “false accepts” while a small number of “false rejects” are permitted per Gage Tolerancing Policy ASME Y14.43, “Absolute Tolerancing (Pessimistic Tolerancing)”. See clause paragraph (d)(2).

• “Maintenance” of AIE refers to keeping all inspection equipment properly maintained and calibrated per ISO 10012 or ANSI/NCSL Z540.3. See clause paragraph (j).

(b) Scope. This clause establishes requirements for design, supply, performance, and maintenance of AIE used for product inspection and acceptance. In addition, this clause establishes requirements for the preparation, submission, and approval of AIE documentation.

Second sentence: In addition, this clause establishes requirements for the preparation, submission, and approval of AIE documentation.

• The MSE DID outlines in detail how the contractor is to prepare, what to include and how to submit its AIE package to the Government for review and approval.

• The MSE DID also outlines requirements for Go-No Go/Functional gages, Standard Measuring Equipment (SME), Special Inspection Equipment/Methods, Coordinate Measuring Machines (CMMs), Tool Control (TC), Non-Destructive Testing (NDT), Automated Acceptance Inspection Equipment (AAIE) & Complex Designs, Destructive, Environmental, and Functional Test Equipment, and Measurement Systems Analysis (MSA).

NOTE 1

Measurement systems that will be utilized for product inspection and acceptance shall complement the requirements of an ISO 9001- 2008 or equivalent Quality Management System (QMS) as well as all contract quality requirements.

For example, ISO 9001-2008 Paragraph 7.6 incorporates “Control of monitoring and measuring equipment” as one of its elements.

Thus, it is appropriate for the Contractor to maintain an effective AIE system that operates within the ISO requirements to provide evidence of conformity of product to TDP and/or contract quality requirements.

Where necessary to ensure valid results, measuring equipment shall (see paragraph 7.6, ISO 9001:2008)

a. be calibrated or verified, or both, at specified intervals, or prior to use, against measurement standards traceable to international or national measurement standards; where no such standards exist, the basis used for calibration or verification shall be recorded (see

ISO 9001:2008 4.2.4);

b. be adjusted or re-adjusted as necessary;

c. have identification in order to determine its calibration status;

d. be safeguarded from adjustments that would invalidate the measurement result;

e. be protected from damage and deterioration during handling, maintenance and storage.

The above excerpt is taken from ISO 9001:2008, Section 7.6 on page 11, with the permission of ANSI on behalf of ISO.

(c) ISO 2014 - All rights reserved

NOTE 2

• The contractor is responsible for the design, calibration, and maintenance of AIE and to make it available to Government personnel for verification and acceptance inspection purposes.

• AIE should be designed to maximize simplicity of operation, and reliability, while minimizing requirements for maintenance, calibration, and special operator skills.

Paragraph (c) AIE

Each sentence of paragraph “(c) AIE” of the clause is discussed and elaborated on below.

First sentence: The Contractor shall provide all AIE necessary to ensure conformance of components and end-items to contract requirements.

Acceptance Inspection Equipment and Automated Acceptance Inspection Equipment are defined as follows in paragraph (a) the MSE Clause:

(a)(1) Acceptance Inspection Equipment (AIE). All equipment

(includes AAIE defined below), special and standard, including dimensional gages, measuring equipment, test fixtures, electronic and physical test equipment, and other test equipment used for examination and test of a product to determine conformance to the Technical Data Package (TDP) which may include drawings and specifications (e.g., Detail, Performance, Weapon specifications, and QAPs).

(a)(2) Automated Acceptance Inspection Equipment (AAIE). AIE in which the inspection and acceptance determination of the product is performed, in whole or in part, in an automatic manner.

AAIE is a subset of AIE. It is the Contractor’s responsibility to ensure all AIE documentation submitted to the Government for review and approval contain all relevant information that will “ensure conformance of components and end-items to contract requirements”.

(c) AIE. The contractor shall provide all AIE necessary to ensure conformance of components and end-items to contract requirements. AIE shall include inspection, measuring, and test equipment whether Government furnished or contractor furnished (including commercially acquired) along with the necessary specifications and procedures for their use (see ISO 10012, paragraph 6.2.1). The AIE shall not create or conceal defects on the product being inspected. All AIE documentation shall contain sufficient information to permit evaluation of the AIE’s ability to test, verify, and/or measure the applicable characteristics or parameters (see DI-QCIC-81960).

Second sentence: AIE shall include inspection, measuring, and test equipment whether Government furnished or Contractor furnished (including commercially acquired) along with the necessary specifications and procedures for their use (see ISO 10012, paragraph 6.2.1).

The AIE submitted to the Government for review and approval may consist of the following as defined in the MSE Clause:

1) (a)(3) Contractor Inspection Equipment. Government-approved equipment utilized by the contractor to perform examination and tests to assure conformance to contract requirements.

There are times when a characteristic being inspected may require a unique piece of AIE to be designed and produced by the contractor.

2) (a)(4) Commercial Inspection Equipment. Industry-developed inspection equipment of universal application, without limitations to a specific part or item, which is advertised or cataloged as available to the trade or to the public on an unrestricted basis at an established price.

Examples would include Coordinate Measuring Machines (CMMs) or Conventional Radiographic Inspection Systems.

(a)(4)(i) Standard Test Equipment. Multiusage equipment that is specific to a function rather than to an item. It includes such items as hardness testers, tensile strength testers, meters, weighing devices, standard gear testers, ohmmeters, voltmeters, and oscilloscopes.

(a)(4)(ii) Standard Measuring Equipment (SME).

Multipurpose equipment and standards used for performing measurements. It includes such items as micrometers, rulers, tapes, height gages, and protractors, etc. Standards include visual inspection equipment such as scratch and dig standards, surface finish comparator, color standards (FED-STD-595), etc.

3) Government designs or Government furnished gages. It is important for the contractor to include Government furnished gages or designs as part of the AIE submission.

The Government needs to maintain records of what inspection equipment is being used to inspect each characteristic.

All AIE submitted must be accompanied by any corresponding specifications and detailed procedures for use. See ISO 10012, paragraph 6.2.1.

Third sentence: The AIE shall not create or conceal defects on the product being inspected.

This may sound straight forward, but there have been occasions where the AIE has actually created or concealed defects from view on the product being inspected. This is unacceptable.

The contractor is to take precautions to prevent the product from being damaged by the AIE and to ensure all defects are revealed.

Fourth sentence: All AIE documentation shall contain sufficient information to permit evaluation of the AIE’s ability to test, verify, and/or measure the applicable characteristics or parameters (see DI-QCIC-81960).

The AIE documentation submitted to the Government is to contain all information necessary to enable Government personnel to properly evaluate the ability of the AIE to test, verify, and/or measure the applicable characteristics or parameters. This may include logical flow charts and other additional data that explains the logic/algorithm used by the AIE to make a determination. In addition, Government AIE reviewers may request to contact the contractor’s personnel responsible for preparing and submitting the AIE package to discuss areas that need further clarification and detail.

Once again, it is encouraged for Government and contractor personnel to work cooperatively reviewing an AAIE package submission. This will accelerate the AIE review process.

NOTE

During MSE Industry Day of 10-11 December 2013, suppliers inquired what is meant by words such as “necessary” and “sufficient” in the clause language and how detailed AIE submissions should be.

The clause language is consistent with language used in commercial standards such as ISO 10012. The team agreed to reference paragraph 6.2.1 of ISO 10012 in the clause paragraph, included below for convenience:

The excerpt above is taken from ISO 10012:2003, Section

6.2.1 on page 4, with the permission of ANSI on behalf of ISO. (c) ISO 2014 - All rights reserved

In addition, the team explained that the DID and inspection requirements within the rest of the MSE clause address what information is needed for AIE submissions.

INTERNATIONAL STANDARD

ISO 10012 First edition 2003-04-15 Measurement management systems — Requirements for measurement processes and measuring equipment

6.2.1 Procedures

Measurement management system procedures shall be documented to the extent necessary and validated to ensure the proper implementation, their consistency of application, and the validity of measurement results.

New procedures or changes to documented procedures shall be authorized and controlled. Procedures shall be current, available and provided when required.

Guidance

Technical procedures may be based on published standard measurement practices, or on customers’ or equipment manufacturers’ written instructions.

Paragraph (d) AIE Designs & Government Furnished Gages

Various AIE designs are employed to inspect technical data package (TDP) requirements. Most designs are contractor designs; however Government designs, which may include source and vendor item control drawings (see ASME Y14.24), exist for particular programs. AIE submissions should have inspection procedures and theory of operation with a level of detail to demonstrate capability of the proposed AIE to perform the required inspection. This may include drawings, work instructions, operation manuals, etc. Government-provided designs have the necessary information for fabrication and use

(d) AIE Designs & Government Furnished Gages. AIE designs are of two types – Government designs (see (d)(1)) and contractor designs (see (d)(2)). When applicable, Government designs or Government furnished gages are designated in the TDP/contract;

responsibility for all other AIE is assigned to the contractor.

The designs, associated inspection procedures, and theory of operation shall have the level of detail to demonstrate capability of the proposed AIE to perform the required inspection.

(1) Government AIE Designs. Government AIE designs may consist of detailed drawings necessary for the fabrication and use of the AIE. Unless otherwise specified, the contractor may submit alternate or modified contractor designs of Government AIE designs.

(2) Contractor AIE Designs. Contractor AIE design drawings shall meet the requirements of ASME Y14.100, ASME Y14.5 and ASME Y14.43 and may include commercial inspection equipment. [“Commercial inspection equipment” is defined as shown in paragraph (a)(4) above. It shall be fully described by catalog listings or other means which provide sufficient information to permit identification and evaluation by the Government and may include illustrations and engineering data.] Designs shall be submitted for any special fixture(s) to be used. Unless otherwise specified, Gage Tolerancing Policy shall be in accordance with ASME Y14.43, “Absolute Tolerancing (Pessimistic Tolerancing).”

(3) Visual Inspection. Visual inspection standards used for the acceptance/rejection of product shall be submitted for approval.

of the AIE. Contractor designs require compliance to ASME Y14.100, ASME Y14.5, and ASME Y14.43. Commercial inspection equipment, as noted in catalogs and description sheets, is also employed as AIE. For acceptance purposes, AIE designs should be in accordance with ASME Y14.43’s Absolute Tolerancing Policy AKA Pessimistic Tolerancing. The definition for Absolute Tolerancing from ASME Y14.43 is as follows:

4.3.1 Absolute Tolerancing (Pessimistic Tolerancing).

Absolute tolerancing (pessimistic tolerancing) is a policy of tolerancing gages that ensures complete random assemblability of parts by applying gagemakers’ tolerances, wear allowances, measurement uncertainties, and form controls, all within the workpiece limits of size and geometric control. Gage tolerances add material to the gaging element, beginning at the limit [e.g., MMC or virtual condition (MMC concept) of the feature being gaged]. Gages produced under this policy will accept most part features that are within tolerance, reject all part features not within tolerance, and reject a small percentage of borderline part features that are technically within tolerance.

Government Furnished Equipment (GFE) will be provided as necessary. These consist of gages to be used to inspect for specific TDP requirements. For example, the Navy provides interface gages for inspection of some of their characteristics.

See paragraph (o).

All submitted Contractor designs and commercial equipment should provide enough detail to demonstrate the AIE’s ability to accurately inspect the desired characteristic(s).

Various methods of visual inspection are used for acceptance of product. When visual inspection is used, visual standards are established and submitted to the Government for review and approval. For example, pictures or hardware standards can contain the defect criteria for visual comparison purposes.

Paragraph (e) AIE Package Submittals

Designs for Approval

Contractor designs and/or the submission for the use of Government designs shall be approved by the Government.

Partial submission of AIE designs is permissible in order to expedite the approval process; however, the response date for design review will be based on the date of the final complete submission of designs.

The contractor may use their own AIE designs or Government AIE designs for inspection of product. However, if Government AIE designs are to be used, these must be formally submitted by the contractor for approval.

Correspondence in English

All AIE correspondence issued to the United States Government shall be made in the English language. However, the Contractor is free to submit the correspondence in the

(e) AIE Package Submittals. The contractor shall prepare the AIE package submittal in accordance with DI-QCIC-81960 in the applicable Contract Data Requirements List (CDRL – DD Form 1423). In addition, the contractor shall adhere to the following requirements:

(1) Designs for Approval. Contractor designs and/or the submission for the use of Government designs shall be approved by the Government. Partial submission of AIE designs is permissible in order to expedite the approval process; however, the response date for design review will be based on the date of the final complete submission of designs.

(2) Correspondence in English. The contractor shall ensure all AIE correspondence and documentation are submitted in English.

(3) Units of Measurement. The units of measurement within the AIE package submittal shall be consistent with the requirements of the Technical Data Package (TDP).

(4) AIE Flow Down. The contractor shall flow down AIE requirements to sub-contractors at any tier who are performing acceptance inspections.

original language along with an English translation in the event translation problems arise.

The objective is to maintain documentation control and avoid potential problems that can occur when translating technical procedures, operation instructions, etc. into English that may affect the original intent of the writings.

Units of measurement

All units of measurements that are applicable to AIE submissions shall be made in accordance with the Technical Data Package (TDP). Government TDPs may be in accordance with the English or Metric systems of measurements. It is the Contractor’s responsibility to convert the units of measurement so they can be consistent with those called out in the TDP.

The objective is to avoid potential problems that can occur when converting units of measurement that may affect the accept/reject decision making process when reviewing the AIE submissions.

AIE Flow Down

It is the Prime Contractor’s responsibility to flow down all AIE requirements to sub-Contractors, gather all applicable AIE from sub-Contractors, and to prepare the AIE package and submit it to the Government for review and approval. The prime contractor can facilitate Government communications with sub-Contractors on a case by case basis for the purposes of clarification of AIE submissions.

Most importantly, AIE requirements apply where inspections are being conducted for the defined Critical, Major, Minor characteristics or special tests.

Paragraph (f) Characteristics for Inspection

The contractor will provide the AIE documentation corresponding to all Critical, Special, and Major characteristics as listed in the item’s TDP specification and/or drawings. The proposed AIE will be submitted for approval per the CDRL. AIE documentation used for Minor characteristic inspections shall also be submitted for approval as required by the selected option:

• (f)(1) AIE documentation will be submitted for inspection of all minor characteristics listed in the TDP drawings and/or specifications. NOTE: This may include 200 series listed minors from specifications or minors called out in drawings

• (f)(2) AIE documentation will be submitted for inspection of those minor characteristics (see note above) selected by the Government.

• (f)(3) This option does not require AIE documentation to be submitted for approval.

Unlisted characteristics are not applicable to this clause.

(f) Characteristics for Inspection. AIE documentation for Critical, Special, and Major characteristic inspections shall be submitted to the Government for approval in accordance with (IAW) the CDRL (DI-QCIC-81960). AIE for Minor characteristic inspections shall be submitted to the Government for approval IAW CDRL (DI-QCIC-81960) and as required below:

(1) □ Listed Minor (characteristics displayed on specifications and/or drawings)

(2) □ Government selected list (as attached or as provided herein)

(3) □ Not submitted

Paragraph (g) Automated Acceptance Inspection Equipment

(g) Automated Acceptance Inspection Equipment. The AAIE shall accept only conforming material. All characteristics requiring AAIE per the TDP shall utilize inspection equipment with a minimum demonstrated reliability of 99.8% at a 90% confidence level to detect non-conforming material unless otherwise specified below.

(1) Reliability of _____% at a _____% Confidence Level for

Critical/Special Characteristics

(2) Reliability of _____% at a _____% Confidence Level for Major

Characteristics

(3) For inspection of major and minor characteristics where contractor utilizes AAIE when it is not required by the TDP, the AAIE package shall be submitted to the Government for approval. If the Minor characteristic is not listed in paragraph (f)(2) or not required for submittal in paragraph (f)(3), then the AAIE requirements (e.g., verification, calibration, prove-out, etc.) of the inspection shall still be performed.

(4) All AAIE packages submitted to the Government for approval shall be in accordance with MIL-A-70625 (Automated Acceptance Inspection Equipment Design, Testing and Approval of). Furthermore, the contractor shall be responsible for producing the acceptance and rejection verification standards/masters representative of the characteristics the AAIE is designed to inspect. The verification standards and frequency of use require Government approval prior to use. When verification standards are used for the VL-VII “sampling plan” per MIL-STD-1916 paragraph 4.4, verification standards and frequency of use shall require Government approval prior to use.

(5) If the AAIE accepts a critical characteristic “reject” standard the contractor shall notify the Government and act in accordance with paragraph (f) of the Critical Characteristic Control Clause. In addition, if the AAIE accepts a major and/or minor characteristic “reject” standard the contractor shall act in accordance with paragraph 8.3 of ISO 10012 or paragraph 5.2.3 of ANSI/NCSL Z540.3.

(6) All AAIE shall be required to pass a Government-approved Acceptance (Prove-Out) Test. The contractor shall conduct this test per the approved test plan and shall submit a test analysis report for approval. See applicable CDRL (DI-QCIC-81960). This test shall be performed at the contractor’s facilities whose manufacturing system has had the AAIE fully integrated and calibrated as per paragraph (j) of this clause. The contractor shall allow Government personnel access to this facility and unobstructed monitoring of this test.

(7) The contractor shall notify the Government prior to a modification and/or relocation of the Government-approved AAIE. The modified AAIE designs shall be submitted for approval. The modified and/or relocated AAIE shall require submission of the acceptance test plan (prove-out) and results for review and approval prior to use. The modified and/or relocated AAIE shall be in accordance with paragraphs (g)(1) – (g)(6).

Paragraph (a)(2) of the clause defines Automated Acceptance Inspection Equipment (AAIE) as “AIE equipment in which the inspection and acceptance determination of the product is performed, in whole or in part, in an automatic manner.”

AAIE can be used to inspect any product characteristic as directed by TDPs. Nevertheless, AAIE is mostly used for inspecting critical and major characteristics and automatically accepting or rejecting the product accordingly. Since the accept/reject decision is made in an automatic manner without human intervention then it is extremely important for the AAIE to be properly proved-out.

The first sentence of the MSE clause states, “The AAIE shall accept only conforming material.” This refers to the actual Automated Acceptance Inspection Equipment (AAIE). The intent of the sentence is to convey the idea that the Government expects the AAIE to work as required when in use by accepting only conforming material and never accepting nonconforming material or rejection verification standards. There have been instances where the submitted AAIE designs were functional in theory and on paper, but the actual AAIE system did not perform as desired during the Government-approved prove-out test or when integrated with the actual production line.

The second sentence of introductory paragraph (g) reads:

All characteristics requiring AAIE per the TDP shall utilize inspection equipment with a minimum demonstrated reliability of 99.8% at a 90% confidence level to detect non-conforming material unless otherwise specified below.

This translates into an AAIE correctly identifying a “reject” standard 1151 times. This is also mentioned in the Review Guide for Critical Characteristics Control (CCC) Plan and Critical Plan of Action (CPOA).

NOTE 1: The use of variable data analysis or a “K factor,” as used in the CCCP/CPOA review guide, are not applicable to reduce the prove-out quantity of non-conformances, e.g., the default requirement of 99.8% reliability at a 90% confidence level still requires the AAIE to properly identify a nonconformance 1151 times. A “K factor” may be used to demonstrate reliability as part of other contractual requirements, e.g., CCCP, but only after the

AAIE prove-out has been successfully completed. The accuracy of the inspection equipment does not guarantee that the AAIE, when integrated into the manufacturing system, will function properly.

Clause paragraphs (g)(1) and (g)(2) of the MSE Clause provide the program offices with the necessary flexibility, up-front, to fill in the desired reliability and confidence levels for the AAIEs associated with their particular programs if the default (minimum demonstrated reliability of 99.8% at a 90% confidence level) is not desired. Reliability and confidence level requirements are intended to assure that only conforming material is presented to the Government for acceptance. They are NOT intended to imply that the Government is willing to accept any non-conforming material.

The reliability of the AAIE needs to be monitored on a continual basis to ensure non-conforming product is never accepted. MIL- A-70625 is the AAIE standard. Other Government specifications such as MIL-A-48078 (maximum error rate of 1/500) and contractual requirements such as the Critical Characteristics Control Clause (Six Sigma) also stress the importance of system reliability.

Clause paragraph (g)(3) is a new sub-paragraph that was created as a result of MSE Industry Day during 10-11 December 2013. It was added to address what to do if a contractor uses AAIE, as a continuous improvement effort, for inspection of Major or Minor characteristics although it is not required per the TDP. The AAIE package will still need to be submitted to the Government for review and approval. In these situations, it is not the intent of the Government to discourage continuous improvement efforts or application of LEAN production practices. Standard calibration, process control, and Measurement System Analysis (MSA) still apply. However, the MSE clause does not stipulate default reliabilities in such cases. Best practices of ISO 10012, ANSI Z540.3, and ASTM E2782-11 still apply.

Clause paragraph (g)(4) states that all AAIE submitted to the Government for approval shall be in accordance with MIL-A-70625 (Automated Acceptance Inspection Equipment Design, Testing and Approval of). If the contractor proposes alternate methods to achieve compliance with MIL-A-70625, then these shall be submitted to the Government for review and approval.

Additionally, paragraph (g)(4) requires the contractor to produce the acceptance and rejection verification standards/masters that will represent the characteristics the AAIE is designed to inspect. Effective standards are designed to include “borderline” conforming and non-conforming product.

The Government will be given assurances of the sensitivity of the AAIE to properly identify and segregate good product from bad. It is these standards that will be used to “prove-out” the AAIE in a Government approved acceptance test to determine if the minimum reliability and confidence level requirements can be met.

NOTE 2: Ideally the verification standards should be fabricated from actual product created by the contractor’s manufacturing processes.

NOTE 3: Additional configurations of standards may be required during prove-out to ensure the AAIE is capable of detecting the specified non-conformances in multiple configurations of the product or for other technically based reasons. For example, the specification lists three different critical defects. Therefore, three individual reject standards will be required to represent each critical defect independently. Additional standards may be required to represent two critical defects present at once, or all three critical defects present at once, or different combinations of the critical defects, etc. The need for additional standards shall be determined on a case by case basis depending on the technicality of the defects or the inspection methodology.

NOTE 4: Additional standards or production parts may be required in order to conduct AAIE accuracy testing. These parts shall be capable of withstanding repeated measurements without degrading. There shall be sufficient parts that represent the expected range of measurement values the AAIE was designed to inspect.

Proper planning for the design, production, testing and implementation of standards shall be incorporated into the contractor’s program schedule. To ensure adequate data is collected, the standards and their frequency of use should be described to the Government in AIE submissions IAW with DI-QCIC- 81960. The contractor shall recalibrate the calibration / verification standards and/or AIE as required.

Paragraph 4.4 (displayed below for convenience) of MIL-STD-1916 requires, “for each Critical Characteristic to implement an AAIE or a failsafe manufacturing operation and apply sampling plan VL-VII to verify the performance of the screening operation.”

4.4 Critical characteristics. Unless otherwise specified in

the contract or product specifications, the contractor is required for each critical characteristic to implement an automated screening or a fail safe manufacturing operation and apply sampling plan VL-VII to verify the performance of the screening operation. The occurrence of one or more critical nonconformances requires corrective action as specified in paragraph 4.5.

If the AAIE is inspecting for critical characteristics, then the performance of the accept/reject decision making operation must be verified using a MIL-STD-1916 VL-VII sampling plan during production – all contractors shall plan accordingly. The VL-VII sampling plan does not alleviate the contractor from following all approved calibration and verification requirements per paragraph (j) of the clause. The verification of the critical characteristic could entail use of a second AAIE, which requires Government approval. However, contractor-proposed provisions (MIL—STD-1916, paragraphs 4.1 & 5.1) may be used in lieu of the VL-VII with Government approval. Keep in mind that all calibration/verification standards and their frequency of use during production to verify the AAIE, as well as the inspection method(s) and equipment used to perform the VL-VII inspection, require Government approval prior to use.

Clause paragraph (g)(5) addresses the issue of what a contractor is to do if their AAIE incorrectly accepts a critical defect master, which is designed to simulate actual defective product.

If such a scenario occurs, the contractor is to follow the directions of paragraph (f) of the SMCA Critical Characteristic Control Clause, which has been copied and pasted below for convenience.

ES6550

CRITICAL CHARACTERISTICS (SIX SIGMA)

52.246-4553 May 2010

(f) In the event that a critical non-conformance is found anywhere in the production process, the contractor, as part of its CCC Plan, shall have procedures in place to ensure:

(1) The non-conformance is positively identified and segregated to ensure that nonconforming product does not inadvertently remain in or reenter the production process. This control shall be accomplished without affecting or impairing subsequent non-conformance analysis. Final disposition of non-conforming product shall be documented and audited for traceability.

(2) The operation that produced the non-conforming component or assembly and any other operations incorporating suspect components or assemblies are immediately stopped. (See para h.

for exceptions)

(3) The government (PCO) is immediately notified of the critical non-conformance (electronic mail)(DI-SAFT-80970A).

(4) Any suspect material is identified, segregated and suspended from any further processing and shipment.

(5) An investigation is conducted to determine the root cause of the non-conformance and the required corrective actions. An evaluation shall also be conducted with regard to suspect material to ensure that no additional critical non-conformances are present. A report of this investigation shall be submitted to the government (DI-SAFT-80970A). The use of the DID report shall not delay notification to the government as required in f(3) above.

(6) A request to restart manufacturing or to use any suspect material associated with the critical non-conformance is submitted to the government (DI-SAFT-80970A). Restart of production shall not occur until authorized by the PCO, unless previously addressed in the approved CCC Plan. The Government will respond to a restart request within 3 working days. All objective evidence of the investigations to date shall be available for review at the time of restart. Suspect material shall not be used without PCO approval.

(7) The procuring activity reserves the right to refuse acceptance of any suspect material until the root cause or reasonably likely cause of the critical non-conformance has been identified, corrective action has been fully implemented and sufficient evidence has been provided to exclude non-conforming material from the conforming population.

Furthermore, paragraph 8.3 of ISO 10012 and paragraph 5.2.3 of ANSI Z540.3-2006 are referenced that detail actions to take if the AAIE is found to be suspect, e.g. accept nonconforming product such as a critical defect master. These paragraphs have also been copied and pasted below for convenience.

The excerpt above is taken from ISO 10012:2003, Section 8.3 on page 13, with the permission of ANSI on behalf of ISO.

(c) ISO 2014 - All rights reserved

INTERNATIONAL STANDARD

ISO 10012 First edition 2003-04-15 Measurement management systems — Requirements for measurement processes and measuring equipment

8.3 Control of nonconformities

8.3.1 Nonconforming measurement management systems

The metrological function shall ensure the detection of any nonconformities, and shall take immediate action.

8.3.2 Nonconforming measurement processes

Any measurement process known to give, or suspected of producing, incorrect measurement results shall be suitably identified and shall not be used until appropriate actions have been taken.

If a nonconforming measurement process is identified, the process user shall determine the potential consequences, make the necessary correction, and take the necessary corrective action.

A measurement process modified due to a nonconformity shall be validated before use.

Guidance

Nonconforming elements should be identified to prevent inadvertent use.

Interim actions (e.g. workaround plans) may be used until the corrective actions have been implemented.

The excerpt above is taken from ANSI/NCSL Z540.3-2006, Section 5.2.3 on page 5, with the permission of ANSI/NCSL - All rights reserved

NOTE 5: When verification standards are used during the verification process in production and the AAIE accepts a critical characteristic “reject” standard, the Government shall be notified.

ANSI/NCSL Z540.3-2006

American National Standard for Calibration – Requirements for the Calibration of Measuring and Test Equipment NCSL International

5.2.3 Nonconforming measuring and test

equipment

Measuring and test equipment that are included in the calibration system and suspected or known:

a) to have been damaged, overloaded, or mishandled;

b) to have malfunctioned in such a way that may invalidate its intended use;

c) to produce incorrect measurement results;

d) to have been used beyond its designated calibration interval without an authorized temporary extension;

e) to have damaged, broken, bypassed, or missing seals or adjustment access controls;

or 1

f) to have been exposed to influencing quantities that can adversely affect its intended use;

shall be removed from service and identified by prominent labeling or marking. Such equipment shall not be returned to service until the reasons for its nonconformity have been investigated and resolved, and it has been recalibrated.

According to clause paragraph (g)(6), all AAIE is required to pass a Government-approved Acceptance (Prove-Out) Test. Prove-out tests per MIL-A-70625 include evaluations of the AAIE for accuracy, reliability, repeatability, and fail-safe. Accuracy testing involves repeated measurements of masters or standards with known values to evaluate the AAIE’s accuracy and the influence of external elements. Reliability tests typically include cycling of “borderline” accept/reject standards intermixed with conforming product over a simulated production run. Repeatability tests include running the defect masters through the AAIE system in series. Fail safe testing includes examining the automated system operation during potentially problematic circumstances, e.g. emergency shutdown, varied start up procedures, loss of air pressure, incomplete inspection cycle, etc. to ensure all product is properly inspected and handled when a non-conformance is discovered.

The contractor will submit a prove-out test plan as part of the AIE submission package (DI-QCIC-81960). The Government is at liberty to recommend modifications to the prove-out test plan as appropriate. After approval of the proposed prove-out test plan, Government personnel shall be granted permission to freely participate in and monitor the prove-out test to validate if the AAIE system reliability meets the contractual requirement(s).

This test shall be performed at the contractor’s facilities where the manufacturing system and the AAIE…