ATTACHMENT 0001 Manufacturing Readiness Level Deskbook V2_May 2011.docx

Manufacturing Readiness Level (MRL) Deskbook

Version 2.0 May, 2011

Prepared by the OSD Manufacturing Technology Program In collaboration with The Joint Service/Industry MRL Working Group

This document is not a DOD requirement and is being offered as a Best Practice

CONTENTS

iii

APPENDICES

A. Detailed MRL Definitions (Threads Matrix) A-1

B. Acronyms .............................................................................................................. B-1

LIST OF FIGURES

LIST OF TABLES

Table 4-1 Example of Added Detail Derived from Site Visits 4-10

Table A-1 Manufacturing Readiness Levels for the Technology and Industrial Base Thread ............................................................................. A-1 Table A-2 Manufacturing Readiness Levels for the Design Thread...................... A-3 Table A-3 Manufacturing Readiness Levels for the Cost and Funding Thread ... A-5 Table A-4 Manufacturing Readiness Levels for the Materials Thread .................. A-8 Table A-5 Manufacturing Readiness Levels for the Process Capability and Control Thread........................................................................................ A-10

Table A-6 Manufacturing Readiness Levels for the Quality Management Thread ..................................................................................................... A-12

Table A-7 Manufacturing Readiness Levels for the Manufacturing Personnel Thread ..................................................................................................... A-13 Table A-8 Manufacturing Readiness Levels for the Facilities Thread ................ A-14 Table A-9 Manufacturing Readiness Levels for the Manufacturing Management Thread.............................................................................. A-15

Executive Summary

Manufacturing status and risk evaluations have been performed as part of defense acquisition programs for years in a variety of forms. These evaluations, while often highly structured and well managed, did not use a uniform metric to measure and communicate manufacturing risk and readiness. They were not conducted on technology development efforts or in early acquisition phases. Furthermore, the frequency of these types of evaluations has declined since the 1990s. Paralleling this decline, manufacturing-related impacts on cost and schedule have grown.

New policy has been established to address this problem in Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System, dated 8 December 2008. It establishes target maturity criteria for measuring risks associated with manufacturing processes at Milestones A, B, and C and Full Rate Production. However, quantitative assessments are necessary to determine whether these criteria have been met.

Manufacturing Readiness Levels (MRLs) and assessments of manufacturing readiness have been designed to manage manufacturing risk in acquisition while increasing the ability of the technology development projects to transition new technology to weapon system applications. MRL definitions create a measurement scale and vocabulary for assessing and discussing manufacturing maturity and risk. Using the MRL definitions, an assessment of manufacturing readiness is a structured evaluation of a technology, component, manufacturing process, weapon system or subsystem. It is performed to:

· Define current level of manufacturing maturity

· Identify maturity shortfalls and associated costs and risks

· Provide the basis for manufacturing maturation and risk management This document provides best practices for conducting assessments of manufacturing readiness. It is designed for acquisition program managers and managers of those technology development projects and pre-systems acquisition technology demonstration projects intending to transition directly to the acquisition community as well as the people who are involved in conducting the assessments.

ES-1

This page is intentionally left blank.

1. Introduction

1.1 MANUFACTURING RISKS RECOGNIZED IN POLICY

Manufacturing status and risk evaluations have been performed as part of defense acquisition programs for years in a variety of forms (e.g. Production Readiness Reviews, Manufacturing Management/Production Capability Reviews, etc.).1 These reviews, while often highly structured and well managed, did not use a uniform metric to measure and communicate manufacturing risk and readiness. They were not conducted on technology development efforts or in early acquisition phases. Furthermore, the frequency of these types of reviews has declined sharply since the 1990s.

Paralleling this decline, manufacturing-related impacts on cost, schedule, and performance have grown. Studies by the Government Accountability Office (GAO) cite a lack of manufacturing knowledge at key decision points as a leading cause of acquisition program cost growth and schedule slippages in major DoD acquisition programs.2 Consequently, policy has been developed to strengthen the way in which manufacturing issues and risks are considered in the defense acquisition system.

There is a long standing policy on manufacturing-related content of acquisition strategies. Defense Federal Acquisition Regulation Supplement (DFARS) Section 207.105b (Contents of Written Acquisition Plans)3 mandates specific national technology and industrial base considerations be included in acquisition strategies for major defense acquisition programs as follows:

· An analysis of the capabilities of the national technology and industrial base to develop, produce, maintain, and support such program, including consideration of factors related to foreign dependency

· Consideration of requirements for efficient manufacture during the design and production of the systems to be procured under the program

· The use of advanced manufacturing technology, processes, and systems during the research and development phase and the production phase of the program

· To the maximum extent practicable, the use of contract solicitations that encourage competing offerors to acquire, for use in the performance of the

1-1 contract, modern technology, production equipment, and production systems (including hardware and software) that increase the productivity of the offerors and reduce the life-cycle costs

· Methods to encourage investment by U.S. domestic sources in advanced manufacturing technology production equipment and processes through: (i) recognition of the contractor’s investment in advanced manufacturing technology production equipment, processes, and organization of work systems that build on workers’ skill and experience, and work force skill development in the development of the contract objective; and (ii) increased emphasis in source selection on the efficiency of production.

Department of Defense Instruction (DoDI) 5000.02 establishes new policy to address manufacturing over the entire life cycle.4 In the Materiel Solution Analysis (MSA) Phase, the policy requires the Analysis of Alternatives (AoA) to assess “manufacturing feasibility.”5 For the Technology Development (TD) Phase, the new policy also affirms that:

· Prototype systems or appropriate component-level prototyping shall be employed to “evaluate manufacturing processes.”6

· A successful preliminary design review will “identify remaining design, integration, and manufacturing risks.”7

· A program may exit the TD Phase when “the technology and manufacturing processes for that program or increment have been assessed and demonstrated in a relevant environment” and “manufacturing risks have been identified.”8 Furthermore, one of the purposes of the Engineering and Manufacturing Development (EMD) Phase is to “develop an affordable and executable manufacturing process.”9 Consequently, the policy goes on to say that: “the maturity of critical manufacturing processes” is to be described in a post-Critical Design Review (CDR) Assessment;10 System Capability and Manufacturing Process Demonstration11 shall show “that system production can be supported by demonstrated manufacturing

10 DoDI 5000.02 Enclosure (2) paragraph 6.c.(6).(c).

11 The second sub-phase of EMD.

processes;”12 and the EMD Phase shall end when “manufacturing processes have been effectively demonstrated in a pilot line environment.”13 Finally, the policy establishes two entrance criteria for the Production and Deployment Phase as “no significant manufacturing risks” and “manufacturing processes [are] under control (if Milestone C is full-rate production).”14 This enables Low Rate Initial Production (LRIP) to result in an “adequate and efficient manufacturing capability”15 so that the following knowledge will be available to support Full-Rate Production (FRP) approval:

· “demonstrated control of the manufacturing process”

· “the collection of statistical process control data”

· “demonstrated control and capability of other critical processes”16

1.2 GUIDANCE ISSUED IN SUPPORT OF POLICY

1.2.1 MANUFACTURING-RELATED SUCCESS CRITERIA ESTABLISHED FOR TECHNOLOGY DEVELOPMENT AND ACQUISITION STRATEGIES

In support of both DFARS language and the new 5000.02, the Defense Acquisition Guidebook17 (DAG) Chapter 2 (Acquisition Program Baselines, Technology Development Strategies, and Acquisition Strategies) provides guidance on including manufacturing capabilities and risks in the Technology Development Strategy (TDS) at Milestone A and the Acquisition Strategy (AS) at Milestones B and C. Both the TDS and the AS are information baselines for efforts that continually evolve during the progression through the acquisition system.

The TDS guides the reduction of technology risk, the determination of the appropriate set of technologies to be integrated into a full system, and the demonstration of critical technologies on representative prototypes. Therefore, the results of the required assessments of manufacturing feasibility carried out in conjunction with the AoA become the basis of meeting the success criteria for the Alternative Systems Review (ASR) and important inputs to the TDS.

The TDS should identify and address how industrial capabilities, including manufacturing technologies and capabilities, will be considered and matured during the TD Phase. Industrial capabilities encompass public and private capabilities to design, develop, manufacture, maintain, and manage DoD products. A discussion of these

12 DoDI 5000.02 Enclosure (2) paragraph 6.c.(6).(d).

13 Ibid.

14 DoDI 5000.02 Enclosure (2) paragraph 7.b .

15 DoDI 5000.02 Enclosure (2) paragraph 7.c.(1).(a).

16 DoDI 5000.02 Enclosure (2) paragraph 7.c.(2).

17 Defense Acquisition Guidebook, Defense Acquisition University, December 17, 2009;

https://dag.dau.mil/Pages/Default.aspx considerations is needed to ensure that the manufacturing capability will be assessed adequately, and that reliable, cost-effective, and sufficient industrial capabilities will exist to support the program’s overall cost, schedule, and performance goals for the total research and development program.

The AS is a comprehensive, integrated plan that identifies the acquisition approach and describes the business, technical, and support strategies that will be followed to manage program risks and meet program objectives. Therefore, the results of the assessments and demonstrations of the technology and manufacturing processes in a relevant environment and the identification of manufacturing risks that are reflected as success criteria for the Preliminary Design Review (PDR) are important inputs to the Industrial Base Capabilities Considerations that are a required part of the AS at Milestone B. Similarly, the results of the demonstrations of manufacturing processes in a pilot line environment that are reflected as success criteria for the Production Readiness Review (PRR) are important inputs to the Industrial Base Capabilities Considerations that are a required part of the AS at Milestone C.

The development of the AS should include results of industrial base capability (public and private) analysis to design, develop, produce, support, and, if appropriate, restart an acquisition program. This includes assessing manufacturing readiness and effective integration of industrial capability considerations into the acquisition process and acquisition programs. For applicable products, the AS should also address the approach to making production rate and quantity changes in response to contingency needs. Consider the following manufacturing threads in developing the strategy:

· Technology and industrial base capabilities

· Design

· Cost and funding

· Materials

· Process capability and control

· Quality management

· Manufacturing personnel

· Facilities

· Manufacturing management

1.2.2 MANUFACTURING-RELATED SUCCESS CRITERIA ESTABLISHED FOR SYSTEMS ENGINEERING REVIEWS

This DoDI 5000.02 policy is specifically reinforced in the DAG Chapter 4 (Systems Engineering) with the establishment of manufacturing-related success criteria for the systems engineering technical reviews that occur prior to the acquisition milestones. In addition, the DAG also contains success criteria developed for the technical review that marks the transition between Integrated System Design18 and System Capability and Manufacturing Process Demonstration. All of these success criteria are presented as questions that should be answered affirmatively.

Success criteria for the ASR19 prior to Milestone A are as follows:

· Have the preliminary manufacturing processes and risks been identified for prototypes?

· Have required investments for technology development, to mature design and manufacturing related technologies, been identified and funded?

· Have initial producibility assessments of design concepts been completed? At the PDR prior to Milestone B the following questions apply:

· Have the majority of manufacturing processes been defined and characterized?

· Are initial manufacturing approaches documented?

· Have producibility assessments of key technologies been completed?

· Has a production cost model been constructed?

· Can the industrial base support production of development articles?

· Have long-lead and key supply chain elements been identified?

Exit questions for the CDR prior to System Capability and Manufacturing Process Demonstration include:

· Have the critical manufacturing processes that affect the key characteristics been identified and their capability to meet design tolerances determined?

· Have process control plans been developed for critical manufacturing processes?

· Have manufacturing processes been demonstrated in a production representative environment?

· Are detailed trade studies and system producibility assessments underway?

18 The first sub-phase of EMD.

19 Only the PDR and the CDR are required by policy.

· Are materials and tooling available to meet pilot line schedule?

· Has the system production cost model been updated, allocated to subsystem level, and tracked against targets?

· Are long-lead procurement plans in place and has the supply chain been assessed?

The following success criteria are associated with the PRR prior to Milestone C:

· Is the detailed design producible within the production budget?

· Are the production facilities ready and required workers trained?

· Is detail design complete and stable enough to enter low rate production?

· Is the supply chain established and stable with materials available to meet planned low rate production?

· Have manufacturing processes been demonstrated and proven in a pilot line environment?

· Have all producibility trade studies and risk assessments been completed?

· Is the production cost model based upon the stable detailed design and been validated?

1.3 OVERARCHING BEST PRACTICES FOR COMPLYING WITH POLICY AND GUIDANCE

Manufacturing knowledge is necessary to meet DoDI 5000.02 policy requirements and follow the associated DAG guidelines. Manufacturing Readiness Levels (MRLs) and assessments of manufacturing readiness are designed to measure this knowledge. They form the basis for managing manufacturing risk in acquisition while increasing the ability of the technology development projects to transition new technology to weapon system applications.

MRL definitions were developed by a joint DoD/industry working group under the sponsorship of the Joint Defense Manufacturing Technology Panel (JDMTP).20 The intent was to create a measurement scale that would serve the same purpose for manufacturing readiness as Technology Readiness Levels (TRLs) serve for technology readiness—to provide a common metric and vocabulary for assessing and discussing

20 MRL Guide, Joint Defense Manufacturing Technology Panel Manufacturing Readiness Level Working Group, February 2007; https://acc.dau.mil/GetAttachment.aspx?id=109616&pname=file&aid=24176&lang=en-US.

1-6 manufacturing maturity21 and risk. MRLs were designed with a numbering system to be roughly congruent with comparable levels of TRLs for synergy and ease of understanding and use.

MRLs can serve as a helpful knowledge-based standard and shorthand for evaluating manufacturing maturity, but they must be supplemented with expert professional judgment. Such judgment is provided through an assessment of manufacturing readiness—a structured, fact-based evaluation of a technology, component, manufacturing process, weapon system or subsystem using the MRL definitions. The assessment is performed to:

· Define current level of manufacturing maturity

· Identify maturity shortfalls and associated costs and risks

· Provide the basis for manufacturing maturation and risk management (planning, identification, analysis, mitigation, implementation, and tracking)

The use of MRLs in conjunction with assessments of manufacturing readiness is an industry best practice. A number of major DoD weapon system suppliers and Original Equipment Manufacturers (OEMs) have integrated MRLs into their gated technology transition processes to help decide when a technology is mature enough to use in a product design. As a result, prime contractors and other OEMs are making better decisions about which technologies to include in product designs resulting in reduced cost, schedule and performance risk. Some of the most important benefits include:

· Providing a roadmap, developed by industry and government experts, of the steps necessary to address and implement a mature manufacturing process that will significantly increase the probability of producing a product that meets program objectives of cost, schedule, and performance.

· Identifying where manufacturing maturity is not progressing on schedule and providing management with an assessment of the risk of the situation and the appropriate corrective actions.

· Involving manufacturing subject matter experts and all other relevant stakeholders early in the design and development process in accordance with commercial industry best practices.

· Enabling effective communications between government and industry and the prime contractor and its suppliers.

MRLs are not intended to be an absolute requirement for proceeding into the next phase of acquisition. Therefore MRLs should be tailored for the specific

21 The terms manufacturing readiness and manufacturing maturity are used interchangeably through this document.

circumstances a program is facing, used to support fact-based decisions, and integrated into the program’s risk management process.

1.4 PURPOSE AND ORGANIZATION OF THIS DOCUMENT

Based on lessons learned from work done in DoD and industry, this document describes how MRLs should be used in conducting assessments of manufacturing maturity and suggests how such assessments should be carried out by:

1. Acquisition program managers for all programs of record

2. Managers for all technology development projects and pre-systems acquisition technology demonstration projects intending to transition directly to the acquisition community22

3. People who are involved in conducting the assessments

The body of this document contains the information listed below.

· A description of the MRLs (Section 2)

· A description of how manufacturing maturity evolves throughout the acquisition management system (Section 3)

· A description of the process for conducting assessments of manufacturing readiness (Section 4)

· A description of manufacturing risk management and the best practices for managing manufacturing maturation (Section 5)

· A description of suggested contract language for implementing MRLs as part of assessments of manufacturing readiness (Section 6)

· A detailed description of desired levels of manufacturing maturity over the acquisition life cycle by MRL thread (Appendix A)

· A list of acronyms (Appendix B) Additional information, available to industry and government, about the MRL definitions, threads, tutorials, and tools can be found at http://www.dodmrl.com/. This site provides the latest versions of all MRL-related material and has links to short courses and to Air Force training presentations. In addition, training is available on the use of MRLs. The Air Force Institute of Technology has developed a three-day MRL course titled “Assessing Manufacturing Readiness (SYS 213).” The Defense Acquisition University has also embedded MRL training into several of its courses.

22 These technology development/demonstration projects include all basic and applied research, science and technology, component development, and prototype efforts that are transitioning into an acquisition program.

2. Manufacturing Readiness Levels

2.1 INTRODUCTION

The basic goal of all acquisition programs is to put required capability in the field in a timely manner with acceptable affordability and supportability. To be successful, the two key risk areas of immature product technologies and immature manufacturing capability must be managed effectively. Manufacturing readiness metrics in combination with technology readiness metrics can help acquisition program managers deal with these risks. Similarly, these metrics are important to technology development managers because, they can be used to achieve and convincingly demonstrate a level of readiness for technology transition that acquisition program managers will find credible. Understanding and mitigating these risks will greatly increase the probability of technology insertion for the technology development community and ultimately aid in improvements in cost, schedule and performance for programs of record.

MRLs and TRLs measure these risks. TRLs are described in Section 2.2 along with their overall relationship to MRLs. Section 2.3 defines the MRLs and Section 2.4 is a definition of terms. MRL thread definitions are provided in Section 2.5.

2.2 TRLS AND THEIR RELATIONSHIP TO MRLS

TRLs provide a systematic metric/measurement system to assess the maturity of a particular technology. TRLs enable a consistent comparison of maturity between different types of technology. The TRL approach has been used for many years in the National Aeronautics and Space Administration (NASA) and is the technology maturity measurement approach for all new DoD programs. TRLs have been primarily used as a tool to assist in tracking technologies in development and their transition into production. The nine hardware TRLs are defined as follows:

· TRL 1: Basic principles observed and reported

· TRL 2: Technology concept or application formulated

· TRL 3: Experimental and analytical critical function and characteristic proof of concept

· TRL 4: Component or breadboard validation in a laboratory environment

· TRL 5: Component or breadboard validation in a relevant environment

· TRL 6: System or subsystem model or prototype demonstrated in a relevant environment

· TRL 7: System prototype demonstration in an operational environment

2-10

· TRL 8: Actual system completed and “flight qualified” through test and demonstration

· TRL 9: Actual system “flight proven” through successful mission operations Manufacturing readiness and technology readiness go hand-in-hand. MRLs, in conjunction with TRLs, are key measures that define risk when a technology or process is matured and transitioned to a system. It is quite common for manufacturing readiness to be paced by technology readiness or design stability. Manufacturing processes will not be able to mature until the product technology and product design are stable. MRLs can also be used to define manufacturing readiness and risk at the system or subsystem level. For those reasons, the MRL definitions were designed to include a nominal level of technology readiness as a prerequisite for each level of manufacturing readiness.

2.3 MANUFACTURING READINESS LEVEL DEFINITIONS

There are ten MRLs (numbered 1 through 10) that are correlated to the nine TRLs in use. The final level (MRL 10) measures aspects of lean practices and continuous improvement for systems in production.

MRL 1: Basic Manufacturing Implications Identified

This is the lowest level of manufacturing readiness. The focus is to address manufacturing shortfalls and opportunities needed to achieve program objectives. Basic research (i.e., funded by budget activity) begins in the form of studies.

MRL 2: Manufacturing Concepts Identified

This level is characterized by describing the application of new manufacturing concepts. Applied research translates basic research into solutions for broadly defined military needs. Typically this level of readiness includes identification, paper studies and analysis of material and process approaches. An understanding of manufacturing feasibility and risk is emerging.

MRL 3: Manufacturing Proof of Concept Developed

This level begins the validation of the manufacturing concepts through analytical or laboratory experiments. This level of readiness is typical of technologies in Applied Research and Advanced Development. Materials and/or processes have been characterized for manufacturability and availability but further evaluation and demonstration is required. Experimental hardware models have been developed in a laboratory environment that may possess limited functionality.

MRL 4: Capability to produce the technology in a laboratory environment

This level of readiness acts as an exit criterion for the Materiel Solution Analysis (MSA) Phase approaching a Milestone A decision. Technologies should have matured to at least TRL 4. This level indicates that the technologies are ready for the Technology Development Phase of acquisition. At this point, required investments, such as manufacturing technology development, have been identified. Processes to ensure manufacturability, producibility, and quality are in place and are sufficient to produce technology demonstrators. Manufacturing risks have been identified for building prototypes and mitigation plans are in place. Target cost objectives have been established and manufacturing cost drivers have been identified. Producibility assessments of design concepts have been completed. Key design performance parameters have been identified as well as any special tooling, facilities, material handling and skills required.

MRL 5: Capability to produce prototype components in a production relevant environment

This level of maturity is typical of the mid-point in the Technology Development Phase of acquisition, or in the case of key technologies, near the mid-point of an Advanced Technology Demonstration (ATD) project. Technologies should have matured to at least TRL 5. The industrial base has been assessed to identify potential manufacturing sources. A manufacturing strategy has been refined and integrated with the risk management plan. Identification of enabling/critical technologies and components is complete. Prototype materials, tooling and test equipment, as well as personnel skills have been demonstrated on components in a production relevant environment, but many manufacturing processes and procedures are still in development. Manufacturing technology development efforts have been initiated or are ongoing. Producibility assessments of key technologies and components are ongoing. A cost model has been constructed to assess projected manufacturing cost.

MRL 6: Capability to produce a prototype system or subsystem in a production relevant environment

This MRL is associated with readiness for a Milestone B decision to initiate an acquisition program by entering into the Engineering and Manufacturing Development (EMD) Phase of acquisition. Technologies should have matured to at least TRL 6. It is normally seen as the level of manufacturing readiness that denotes acceptance of a preliminary system design. An initial manufacturing approach has been developed. The majority of manufacturing processes have been defined and characterized, but there are still significant engineering and/or design changes in the system itself. However, preliminary design has been completed and producibility assessments and trade studies of key technologies and components are complete. Prototype manufacturing processes and technologies, materials, tooling and test equipment, as well as personnel skills have been demonstrated on systems and/or subsystems in a production relevant environment. Cost, yield and rate analyses have been performed to assess how prototype data compare to target objectives, and the program has in place appropriate risk reduction to achieve cost requirements or establish a new baseline. This analysis should include design trades. Producibility considerations have shaped system development plans. The Industrial Capabilities Assessment (ICA) for Milestone B has been completed. Long-lead and key supply chain elements have been identified.

MRL 7: Capability to produce systems, subsystems, or components in a production representative environment

This level of manufacturing readiness is typical for the mid-point of the Engineering and Manufacturing Development (EMD) Phase leading to the Post- CDR Assessment. Technologies should be on a path to achieve TRL 7. System detailed design activity is nearing completion. Material specifications have been approved and materials are available to meet the planned pilot line build schedule. Manufacturing processes and procedures have been demonstrated in a production representative environment. Detailed producibility trade studies are completed and producibility enhancements and risk assessments are underway. The cost model has been updated with detailed designs, rolled up to system level, and tracked against allocated targets. Unit cost reduction efforts have been prioritized and are underway. Yield and rate analyses have been updated with production representative data. The supply chain and supplier quality assurance have been assessed and long-lead procurement plans are in place. Manufacturing plans and quality targets have been developed. Production tooling and test equipment design and development have been initiated.

MRL 8: Pilot line capability demonstrated; Ready to begin Low Rate Initial Production

This level is associated with readiness for a Milestone C decision, and entry into Low Rate Initial Production (LRIP). Technologies should have matured to at least TRL 7. Detailed system design is complete and sufficiently stable to enter low rate production. All materials, manpower, tooling, test equipment and facilities are proven on pilot line and are available to meet the planned low rate production schedule. Manufacturing and quality processes and procedures have been proven in a pilot line environment and are under control and ready for low rate production. Known producibility risks pose no significant challenges for low rate production. Cost model and yield and rate analyses have been updated with pilot line results. Supplier qualification testing and first article inspection have been completed. The Industrial Capabilities Assessment for Milestone C has been completed and shows that the supply chain is established to support LRIP.

MRL 9: Low rate production demonstrated; Capability in place to begin Full Rate Production

At this level, the system, component or item has been previously produced, is in production, or has successfully achieved low rate initial production. Technologies should have matured to TRL 9. This level of readiness is normally associated with readiness for entry into Full Rate Production (FRP). All systems engineering/design requirements should have been met such that there are minimal system changes. Major system design features are stable and have been proven in test and evaluation. Materials, parts, manpower, tooling, test equipment and facilities are available to meet planned rate production schedules. Manufacturing process capability in a low rate production environment is at an appropriate quality level to meet design key characteristic tolerances. Production risk monitoring is ongoing. LRIP cost targets have been met, and learning curves have been analyzed with actual data. The cost model has been developed for FRP environment and reflects the impact of continuous improvement.

MRL 10: Full Rate Production demonstrated and lean production practices in place

This is the highest level of production readiness. Technologies should have matured to TRL 9. This level of manufacturing is normally associated with the Production or Sustainment phases of the acquisition life cycle. Engineering/design changes are few and generally limited to quality and cost improvements. System, components or items are in full rate production and meet all engineering, performance, quality and reliability requirements. Manufacturing process capability is at the appropriate quality level. All materials, tooling, inspection and test equipment, facilities and manpower are in place and have met full rate production requirements. Rate production unit costs meet goals, and funding is sufficient for production at required rates. Lean practices are well established and continuous process improvements are ongoing.

Although the MRLs are numbered, the numbers themselves are unimportant. The numbers represent a non-linear ordinal scale that identifies what maturity should be as a function of where a program is in the acquisition life cycle (as described in Section 3). Using numbers is simply a convenient naming convention.

2.4 DEFINITION OF TERMS

As manufacturing readiness increases, demonstration of manufacturing capabilities should be accomplished in more realistic environments. Prior to Milestone A, the MRLs focus on manufacturing feasibility by identifying and reducing the production risk of the proposed concepts. These proposed technology concepts are generally demonstrated in a laboratory environment. MRLs focus on identifying manufacturing challenges that should be addressed in the TD phase.

Prior to Milestone B, MRLs focus on a contractor’s capability to produce prototypes in a production relevant environment, outside of the laboratory. The parameters defining a production relevant environment should be based on the risks and uniqueness associated with demonstrating that contractors’ key processes meet program requirements.

A production relevant environment represents the manufacturing capability needed to proceed into the EMD Phase with high confidence of achieving program cost, schedule and performance requirements. This level of production realism is well beyond what is seen in a laboratory. The emphasis is on addressing higher risk areas (e.g. more advanced technologies and newer manufacturing capabilities). During this critical junction it is essential that the contractor(s) demonstrate the capability to build the product or a similar product (e.g. size, tolerances, quality levels, processes, and testing) in the facility intended to be used during production.

Production relevant environment—An environment with some shop floor production realism present (such as facilities, personnel, tooling, processes, materials etc.). There should be minimum reliance on laboratory resources during this phase. Demonstration in a production relevant environment implies that contractor(s) must demonstrate their ability to meet the cost, schedule, and performance requirements of the EMD Phase based on their production of prototypes. The demonstration must provide the program with confidence that these targets will be achieved, but does not require a production line. Furthermore, there must be an indication of how the contractor(s) intend to achieve the requirements in a production representative and pilot environments.

As a program evolves through the EMD phase and hardware is built for qualification testing, the manufacturing processes should become more robust and mature to address production representative activities on the whole program.

Production representative environment—An environment that has as much production realism as possible, considering the maturity of the design. Production personnel, equipment, processes, and materials that will be present on the pilot line should be used whenever possible. The work instructions and tooling should be of high quality, and the only changes anticipated on these items are associated with design changes downstream that address performance or production rate issues. There should be no reliance on a laboratory environment or personnel.

The final stage of EMD is producing products that look and operate like they are production units from LRIP. These units need to be built on a pilot production line to adequately demonstrate the ability to migrate from EMD to LRIP. Without this realism it would be very difficult to obtain confidence that the production process will be able to meet cost, schedule, and performance (e.g. quality) requirements for production.

Pilot line environment—An environment that incorporates all of the key production realism elements (equipment, personnel skill levels, facilities, materials, components, work instructions, processes, tooling, temperature, cleanliness, lighting etc.) required to manufacture production configuration items, subsystems or systems that meet design requirements in low rate production. To the maximum extent practical, the pilot line should utilize full rate production processes.

The definitions of relevant, representative, and pilot line environments are intended to demonstrate the natural progression of manufacturing maturity throughout the acquisition life cycle. The program office and contractor must reach agreement on the detailed production realism content (equipment, personnel skill levels, processes, etc.) for each definition above. This agreement must be based on the specific situation and its associated manufacturing risk in order to mitigate that risk in a timely and thorough manner.

Two other definitions are germane to this discussion.

Manufacturability—The characteristics considered in the design cycle that focus on process capabilities, machine or facility flexibility, and the overall ability to consistently produce at the required level of cost and quality. Associated activities may include some or all of the following:

· Design for commonality and standardization—uses fewer parts

· Design for environmental and safety compliance

· Design for multi-use and dual-use applications

· Design for modularity and plug compatible interface/integration

· Design for flexibility/adaptability or use “robust design”

· Utilize reliable processes and materials

· Utilize monolithic and determinant assembly

· Design for manufacturing and assembly

· Achieve production yield Producibility—The relative ease of producing an item that meets engineering, quality and affordability requirements. Associated activities may include some of the following:

· Design for specific process capability and control parameters

· Perform material characterization analysis

· Perform variable reduction analysis, e.g., Taguchi and design of experiments

· Develop critical materials and processes before selecting product design

· Utilize modeling and simulation for product and process design tradeoffs

· Design and development of closed-loop process control on critical items

2.5 MRL THREADS AND SUB-THREADS

Successful manufacturing has many dimensions. MRL threads have been defined to organize these dimensions into nine manufacturing risk areas. The threads are as follows:

· Technology and the Industrial Base: Requires an analysis of the capability of the national technology and industrial base to support the design, development, production, operation, uninterrupted maintenance support of the system and eventual disposal (environmental impacts).

· Design: Requires an understanding of the maturity and stability of the evolving system design and any related impact on manufacturing readiness.

· Cost and Funding: Requires an analysis of the adequacy of funding to achieve target manufacturing maturity levels. Examines the risk associated with reaching manufacturing cost targets.

· Materials: Requires an analysis of the risks associated with materials (including basic/raw materials, components, semi-finished parts, and subassemblies).

· Process Capability and Control: Requires an analysis of the risks that the manufacturing processes are able to reflect the design intent (repeatability and affordability) of key characteristics.

· Quality Management: Requires an analysis of the risks and management efforts to control quality, and foster continuous improvement.

· Manufacturing Workforce (Engineering and Production): Requires an assessment of the required skills, availability, and required number of personnel to support the manufacturing effort.

· Facilities: Requires an analysis of the capabilities and capacity of key manufacturing facilities (prime, subcontractor, supplier, vendor, and maintenance/repair).

· Manufacturing Management: Requires an analysis of the orchestration of all elements needed to translate the design into an integrated and fielded system (meeting Program goals for affordability and availability).

Many of the MRL threads have been decomposed into sub-threads. This enables a more detailed understanding of manufacturing readiness and risk, thereby ensuring continuity in maturing manufacturing from one level to the next. For example:

· Technology and the Industrial Base includes industrial base issues and manufacturing technology development

· Design includes producibility and maturity

· Cost and Funding includes production cost knowledge (cost modeling), cost analysis, and manufacturing investment budget

· Materials includes maturity, availability, supply chain management, and special handling (i.e. government furnished property, shelf life, security, hazardous materials, storage environment, etc.)

· Process Capability and Control includes modeling and simulation (product and process), manufacturing process maturity, and process yields and rates

· Quality Management includes supplier quality

· Manufacturing Management includes manufacturing planning and scheduling, materials planning, and tooling/special test and inspection equipment

The matrix shown in Appendix A provides detailed criteria for each of the ten MRLs, by thread and sub-thread, throughout the acquisition life cycle. The matrix allows a user to separately trace and understand the maturation progress of each of the threads and sub-threads as readiness levels increase from MRL 1 though MRL 10. These thread and sub-thread MRL criteria should be applied when appropriate to the situation and may be tailored to a particular technology or application.

As stated earlier, the MRL numbering scheme is not important for assessments of manufacturing readiness. The degree of maturity of an element of a program that is being assessed, whether the target maturity has been achieved, and what has to be accomplished to increase maturity are important. This information is discovered in the assessment process using the matrix in Appendix A, not by assigning a number to the element being assessed.

This page is intentionally left blank

3. MRLs and the Acquisition Management System

3.1 INTRODUCTION

Manufacturing risk management plays an integral part in the acquisition of all weapon systems throughout their entire life cycle.23 MRLs should be used in source selection to assess the manufacturing maturity and risk of each offer. If multiple prototypes are used in a down-select process for the next phase of acquisition, MRL- based assessments should be performed on each configuration to provide critical knowledge of manufacturing maturity and risk of each prototype. Delivering weapon systems in a timely and cost-effective manner is not possible if these risks are not well managed.

Manufacturing risk management is based on an understanding of the reasons why systems did not meet MRL-targets and a determination of the associated impact throughout the life cycle. This effort highlights areas needing management attention and helps ensure successful execution and transition of the program/project24 into the next phase. When targets are not met, the program should develop and implement a Manufacturing Maturation Plan (MMP)25 to ensure that the appropriate level of maturity will be achieved at the next decision point.

While MRLs show a natural progression of manufacturing maturity throughout the acquisition life cycle, the progressions are not all equal. That is why focusing on MRL numbers is a poor practice. There may be significant risks in achieving the next level of maturity even when a program is maturing on schedule. Although assessments of manufacturing readiness assist a program to effectively and efficiently mature the manufacturing process, they must be integrated with program objectives and constraints within the overall systems engineering environment. In addition, MRLs can increase or decrease as a result of changes to the facility, processes, suppliers, design, etc. Such changes do not necessarily mean greater or lesser risk. For example, lowering the current MRL might be driven by implementing a major producibility improvement that will save millions of dollars and even reduce risk.

A common question is the return on investment for conducting MRL-based assessments of manufacturing readiness. The investment to conduct effective assessments and manage the identified risks should be part of a company’s or program office’s standard operating procedures. Unfortunately, the return on that investment is

23 The acquisition life cycle is defined by the acquisition management system.

24 The term “program” refers to an acquisition program of record. The term “project” refers to any technology development effort (ranging from basic research to advanced component development and prototypes) prior to the establishment of a program of record in the acquisition life cycle even though an acquisition program office is often formed prior to that point in time.

25 The MMP addresses the manufacturing risk and provides a mitigation plan for each risk area. See section 5 of this deskbook.

3-10 very difficult to quantify just like any other risk category (e.g., it is not possible to determine a return on investment for a failure modes and effects analysis).