Attachment_0007_DOD_Risk_Issue_and_Opportunity_Mgmt_Guide_Jan2017.pdf

Department of Defense

Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

January 2017

Office of the Deputy Assistant Secretary of Defense for Systems Engineering

Washington, D.C.

Department of Defense Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

Deputy Assistant Secretary of Defense Systems Engineering 3030 Defense Pentagon 3C167 Washington, DC 20301-3030

E-mail: osd.atl.asd-re.se@mail.mil Website: www.acq.osd.mil/se

Distribution Statement A: Approved for public release.

This page is intentionally blank.

Contents

PREFACE

INTRODUCTION ................................................................................................................................... 3 1

Purpose .......................................................................................................................................... 3 1.1

Scope ............................................................................................................................................. 4 1.2

Risk Management Overview ......................................................................................................... 4 1.3

MANAGING RISK BY ACQUISITION PHASE ......................................................................................... 7 2

2.1 Planning Considerations

2.1.1 Strategy Development

2.1.2 Framing Assumptions

2.1.3 Integration with Contractor’s Processes

2.2 Pre-Materiel Development Decision Phase

2.3 Materiel Solution Analysis Phase

2.3.1 Suggested Activities in the MSA Phase to Reduce Risk

2.4 Technology Maturation and Risk Reduction Phase

2.4.1 Suggested Activities and Practices in the TMRR Phase to Reduce Risk

2.5 Engineering and Manufacturing Development Phase

2.5.1 Suggested Activities in the EMD Phase to Reduce Risk

2.6 Production and Deployment Phase

2.6.1 Suggested Activities in the P&D Phase to Reduce Risk

2.7 Operations and Support Phase

3 RISK AND ISSUE MANAGEMENT

3.1 Risk Process Planning

3.2 Risk Identification

3.2.1 Risk Identification Methodologies

3.2.2 Risk Categories

3.2.3 Risk Statement

3.2.4 Evaluation of Candidate Risks

3.3 Risk Analysis

3.3.1 Consequence

3.3.2 Likelihood

3.3.3 Risk Reporting Matrix

3.3.4 Risk Register

vDoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

3.4 Risk Mitigation

3.4.1 Risk Acceptance (and Monitoring)

3.4.2 Risk Avoidance

3.4.3 Risk Transfer

3.4.4 Risk Control

3.4.5 Risk Burn-Down

3.5 Risk Monitoring

3.6 Issue Management

OPPORTUNITY MANAGEMENT .......................................................................................................... 43 4

MANAGEMENT OF CROSS-PROGRAM RISKS ..................................................................................... 49 5

APPENDIX A. PROGRAM RISK PROCESS AND ROLES

A.1 Program Risk Process

A.2 Risk Management Board and Risk Working Group

A.3 Selecting a Risk Management Tool

A.4 Risk Management Roles and Responsibilities A.4.1 Government Responsibilities A.4.2 Typical Contractor Responsibilities A.4.3 Suggested Tiered Roles and Responsibilities

APPENDIX B. RISK MANAGEMENT IN RELATION TO OTHER PROGRAM MANAGEMENT AND

SYSTEMS ENGINEERING TOOLS

B.1 Work Breakdown Structure

B.2 Integrated Master Plans and Integrated Master Schedules

B.3 Earned Value Management

B.4 Technical Performance Measures and Metrics

B.5 Schedule Risk Analysis

B.6 Cost Risk Analysis

B.7 Performance Risk Analysis

APPENDIX C. RISK MANAGEMENT PROCESS VIGNETTE

GLOSSARY

ACRONYMS

REFERENCES

vi DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

FIGURES

Figure 1-1. Overview of Potential Sources of Program Risks, Issues, and Opportunities

Figure 2-1. DoD Acquisition Life Cycle

Figure 2-2. Materiel Solution Analysis Phase Activities

Figure 2-3. Technology Maturation and Risk Reduction Phase Activities

Figure 2-4. Engineering and Manufacturing Development Phase Activities

Figure 2-5. Production and Deployment Phase Activities

Figure 3-1. Risk and Issue Management Process Overview

Figure 3-2. Risk Process Planning

Figure 3-3. Risk Identification

Figure 3-4. Risk Analysis

Figure 3-5. Risk Reporting Matrix and Criteria

Figure 3-6. Risk Matrix Showing Prioritized Results

Figure 3-7. Risk Mitigation

Figure 3-8. Risk Burn-Down

Figure 3-9. Risk Monitoring

Figure 3-10. Example Risk Monitoring and Trend Matrix

Figure 3-11. Suggested Risk Reporting Format

Figure 3-12. Issue Management Process

Figure 3-13. Issue Consequence Reporting Matrix

Figure 4-1. Opportunities Help Deliver Should-Cost Objectives

Figure 4-2. Opportunity Management Process

Figure 4-3. Opportunity Register

Figure 5-1. Sample Synchronization from the SEP Outline

Figure 5-2. Tracking Interdependency Risks

Figure A-1. Government and Contractor Joint Risk Management Boards

Figure A-2. Roles and Responsibilities Tiering

Figure B-1. Example of WBS Levels

Figure B-2. Government and Contractor WBS Relationship

Figure B-3. IMP/IMS Creation and Implementation

Figure B-4. Sample 14-Point Schedule Health Assessment Items and Status viiDoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

Figure C-1. Risk Matrix for Ram Air Turbine Generator

Figure C-2. Risk Burn-Down Diagram for Option A

Figure C-3. Risk Reporting Chart

TABLES

Table 3-1. Sample Consequence Criteria

Table 3-2. Typical Likelihood Criteria

Table 3-3. Weighted Consequence Risk Mitigation

Table 3-4. Risk Register Excerpt

Table 5-1. Sample Table of Required MOAs viii DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

Preface

This guide is one of several Department of Defense (DoD) policy and guidance documents that address the Department’s focus on risk management. It builds from and supersedes the DoD Risk Management Guide but reflects revisions to emphasize managing not only program risks but also issues and opportunities. Readers should assume that general references to managing risk would be tailorable to complementary activities addressing issues and opportunities.

A range of policy, regulatory, or statutory directives call out risk management for its recognized positive relationship to program outcomes. However, the value of risk management is not tied to a formal adherence to policy. Rather the value lies in the Program Manager’s (PM) ability to apply critical thinking and adopt a culture of risk management that influences program decisions and execution of technical fundamentals. This approach aims to manage uncertainty and increase predictable outcomes in delivering capability to the warfighter.

Risk management is an integral part of program management and systems engineering. A PM must align risk appetite with organizational capacity to manage risks and allocate limited resources to the best effect. Risk management principles addressed in this document echo the time-proven 1986 Packard Commission recommendations and reflect recent DoD Better Buying Power initiatives. In 2015, the Under Secretary of Defense for Acquisition, Technology, and Logistics emphasized risk management as a focus of Better Buying Power:

Risk management is an endeavor that begins with requirements formulation and assessment, includes the planning and conducting of a technical risk reduction phase if needed, and strongly influences the structure of the development and test activities.

Active risk management requires investment based on identification of where to best deploy scarce resources for the greatest impact on the program’s risk profile. PMs and staff should shape and control risk, not just observe progress and react to risks that are realized. Anticipating possible adverse events, evaluating probabilities of occurrence, understanding cost and schedule impacts, and deciding to take cost effective steps ahead of time to limit their impact if they occur is the essence of effective risk management. Risk management should occur throughout the lifecycle of the program and strategies should be adjusted as the risk profile changes.

This guide describes strategies and processes for risk, issue, and opportunity (RIO) management that programs should begin early in program development and apply continuously throughout the acquisition life cycle. Each program should tailor the practices and not get bogged down in unnecessary or excess process formality that doesn’t add value. Early identification of the program’s key uncertainties and challenges can inform decisions on the basic program structure and the activities needed to enable successful and efficient delivery of the intended product.

Although this guide focuses primarily on the government program office, industry plays a central role in executing the management necessary for delivery of acquisition products. Government and

1DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

Preface industry may differ in the prioritization of risks, driven in part by differing perspectives or incentives.

Certainly the type of contract, cost or fixed price and associated incentives, can affect the nature of the actions taken by government and industry in their respective roles. Nevertheless, close collaboration and a shared commitment to realism, even when inconvenient, are essential to effective risk management.

The guide is organized as follows:

Section 1: Introduces the scope and overview of the guide.

Section 2: Describes how risk informs the decisions shaping a program acquisition strategy and structure, and the most important activities to manage risk by life cycle phase.

Section 3: Describes how a program manages risks and issues by developing plans to reduce the consequences and/or the likelihood of the risks or issues.

Section 4: Describes opportunity management, including the similarities and differences between opportunity and risk management.

Section 5: Highlights considerations to manage risks related to internal and external interfaces with interdependent programs. Discusses the different priorities of interdependent programs and techniques to manage and mitigate cross-program risks.

Appendixes: Appendix A provides additional information on establishing and documenting a Program Risk Process (PRP). Appendix B discusses integrating risk management with other program management and systems engineering tools, and Appendix C provides a vignette illustrating how a program might address a particular risk.

Some sections contain text boxes with expectations/take-aways that programs should have in mind as they seek to improve the planning and execution of risk management processes and techniques.

2 DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

1 INTRODUCTION

1.1 Purpose

This guide seeks to advance the ability of DoD programs to plan for and manage risks, issues, and opportunities. The quality of thinking and judgment applied to these areas often will determine whether a program meets its objectives throughout the life cycle. Managing these areas requires strategic thinking and begins with early decisions on program structure that take into account the program’s unique uncertainties and risks. The analysis and informed judgment needed to identify and control risk are fundamental to effective program planning and management.

For the purposes of this guide, the terms risk, issue, and opportunity are defined as follows:

• Risks are potential future events or conditions that may have a negative effect on achieving program objectives for cost, schedule, and performance. Risks are defined by (1) the probability (greater than 0, less than 1) of an undesired event or condition and (2) the consequences, impact, or severity of the undesired event, were it to occur.

• Issues are events or conditions with negative effect that have occurred (such as realized risks) or are certain to occur (probability of 1) that should be addressed.

• Opportunities have potential future benefits to the program’s cost, schedule, and/or performance baseline.

Figure 1-1 shows a simple portrayal of technical, programmatic, and business events that may lead to risks, issues, or opportunities, each with cost, schedule, or performance consequences.

Figure 1-1. Overview of Potential Sources of Program Risks, Issues, and Opportunities

Programmatic Events

Business Events

Technical Events

What can go wrong?

What can be improved?

What has or is certain to go wrong?

Issue Management

Risk Management

Consequences:

Both positive and negative impacts to cost, schedule, and performance

Opportunity Management

3DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

1 Introduction

This guide should be used in conjunction with related directives: public law (Title 10 and the Weapon Systems Acquisition Reform Act of 2009); DoDI 5000.02; Defense Acquisition Guidebook (DAG); Military Department guidance; and other instructions, policy memorandums, and regulations.

1.2 Scope

DoD distinguishes mandatory policy from recommended guidance. This document serves solely as guidance for risk management approaches for DoD acquisition programs. It is to be used as appropriate reference material and not as a checklist. This guide reflects experience drawn from numerous DoD programs and suggests risk considerations programs should keep in mind when developing an Acquisition Strategy and program structure, including activities bearing on risk by phase and considerations relating to contract type.

This guide includes a strategic consideration of how risk shapes program structure and content, as well as a suggested process to manage discrete risks by phase. The process is designed to produce risk mitigation plans, which provide the substantive steps a program will take to mitigate its actual risks. This guide uses the term “risk mitigation plan” to refer to the plans that are initially summarized in the program’s Acquisition Strategy and updated as risks are identified and managed.

It also suggests the term Program Risk Process (PRP) to refer to the documented process a program uses to execute its tailored risk, issue, and opportunity processes. The term PRP is not mandatory but is a suggestion to distinguish process documentation from descriptions of risk mitigation plans.

This guide focuses centrally on risk. Analogous discussions, with some distinctions, could apply to issues and opportunities as well. This guide does not attempt to address, in detail, the specific requirements to prevent and manage system safety or system hazards, including environment, safety, and occupational health (ESOH) hazards. The reader should refer to DoDI 5000.02, Enclosure 3 and MIL-STD-882, Standard Practice for System Safety, for guidance in these areas. Likewise, this guide does not attempt to address specialized risks such as cybersecurity. Programs should refer to DoDI 8510.01, Risk Management Framework for DoD Information Technology (IT), for policy and procedures regarding the integrated enterprise-wide structure for cybersecurity risk management.

Programs should develop a method to map these specialized ESOH and cybersecurity risks or issues into their overall risk/issue management processes. Using the forthcoming Directive-Type Memorandum (DTM) 17-001, Cybersecurity in the Defense Acquisition System, programs should consider cybersecurity in technical risk management activities to address risk identification, analysis, mitigation planning, mitigation implementation, and tracking. They should use evolving program and system threats to inform operational impacts.

1.3 Risk Management Overview

The PM is responsible for implementing effective risk management within program constraints.

Successful risk management requires planning and resourcing, and should be implemented early in the life cycle beginning with the Materiel Solution Analysis (MSA) phase or earlier based on early

4 DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

1 Introduction collaboration among the operational, acquisition, and technology communities. The goal is to identify risks to inform decisions on structure and content, and develop mitigation strategies for the risks that must be addressed to deliver intended capabilities.

The practice of risk management constitutes a significant aspect of program management and draws from all disciplines, including systems engineering, use of models and simulation, requirements definition, developmental and operational test, earned value management (EVM), production planning, quality assurance, and logistics. Risk management needs to be both top-down (program leadership) and bottom-up (from working-level staff members) to be successful. PMs should encourage everyone on their program to take ownership of the risk management program and should be careful not to cultivate a “shoot the messenger” culture. All personnel should be encouraged to identify risks, issues, and opportunities and, as appropriate, to support analysis, mitigation, and monitoring activities.

Making risk management work depends on process, but more importantly on people with knowledge and experience in the disciplines relevant to the product, and with the resolve to identify and address the risks that could influence program objectives. An organizational climate, open to external perspectives, that seeks independent board members for design reviews can strengthen the effectiveness of a program’s risk management. Well-understood requirements flowed to the product, an integrated schedule coupled to earned value management (EVM), an independent cost estimate, and the tenacity to pull on the threads that reveal problems all contribute to prospects for success.1

While the processes described in the guide enable risk management, the risk mitigation plans for individual risks (the output of the processes) are significantly more important. In the end, what matters most are the quality and effectiveness of the program’s risk mitigation plans and their implementation in reducing the risks to realizing program objectives, not the process itself.

The steps of the risk management process are generally applicable to the management of risks and issues in multiple phases of the life cycle. At the same time, the nature of the specific actions taken for each step typically will be different depending on the program phase. The differences in the specific actions are driven by the changing types of individual risk, the information and tools available, the outcomes that need to be achieved, the degree of maturity and stability that must be demonstrated, and the residual risks that are tolerable following mitigation efforts.

Programs should define, implement, and document an appropriate, tailored risk process. The process should address planning, identification, analysis, mitigation, and monitoring of risks and issues. See Section 3 for more detail.

1 Drawn from personal communication with VADM Joseph Dyer, June 1993.

5DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

This page is intentionally blank.

2 MANAGING RISK BY ACQUISITION PHASE

Planning Considerations 2.1

2.1.1 Strategy Development

The most important decisions to control risk are made early in a program life cycle (Figure 2-1). PMs and teams1 must understand the capabilities under development and perform a detailed analysis to identify the key risks. During the early phases, the program works with the requirements community to help shape the product concept and requirements. Once the concept and requirements are in place, the team determines the basic program structure, the acquisition strategy, and what acquisition phase to enter based on the type and level of key risks. Risk steers planning and tailoring.

If technology maturity or requirements stability risks exist, the PM should structure a program to enter the life cycle at Milestone A. For example, if there is some doubt as to whether the program can achieve the requirements, the PM should consider a risk reduction phase with competitors building and testing prototypes in order to validate achievability of the requirements. Programs also may use prototypes to quantify the impact of technology on performance, demonstrate the ability to integrate new technologies into mature architectures, and to reduce risk and cost.

If the technologies are mature, the integration of components is at acceptable risk, and the requirements are stable and achievable, the PM can consider entering directly at Milestone B to begin Engineering and Manufacturing Development (EMD).

If a materiel solution already exists and requires only military modification or orientation, the PM can structure the program enter at Milestone C with a small research and development (R&D) effort to militarize the product.

Figure 2-1. DoD Acquisition Life Cycle

1 Early in a program life cycle, a PM may not yet be designated. However, there is usually a responsible acquisition organization overseeing development planning efforts.

Mandatory technical reviews

Best practice technical reviews and audits

Test reviews

BA C

Materiel Development

Decision

CDD

Validation

Development

RFP

Release Decision

IOCFRP FOC = Milestone Decision

= Decision Point

CDRPDRASR SRR SFR SVR/

FCA PRR PCA

TRR

OTRR

Materiel Solution Analysis

Technology Maturation &

Risk Reduction

Operations & Support

DisposalIntegration OT&E

LRIP Production & Deployment

Sustainment

Multiple Software Builds

Engineering & Manufacturing Development

Pr e-

A cq ui si tio n

C on ce pt s, Ex pe rim en ta tio n an d Pr ot ot yp in g

En ab lin g

S& T

7DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

2 Managing Risk by Acquisition Phase

Statute requires PMs to document a comprehensive approach for managing and mitigating risk (including technical, cost, and schedule risk) in the Acquisition Strategy. Likewise, DoD policy requires programs to summarize the top program risks and associated risk mitigation plans in the Acquisition Strategy. Policy also requires programs to describe their technical approach, including key technical risks, and management and execution of risk mitigation activities, in the Systems Engineering Plan (SEP).

2.1.2 Framing Assumptions

When developing the acquisition strategy and program framework, programs make assumptions.

Programs sometimes make assumptions without realizing it, but these framing assumptions need to be identified and validated because they can put the entire program at risk if they turn out to be incorrect. Examples of framing assumptions include priority or achievability of requirements, schedule dependencies, procurement quantities, threats, availability of specialty metals or technology, or accuracy of models and simulations.

2.1.3 Integration with Contractor’s Processes

Risk management is not a stand-alone process. It is integral to other program processes, such as requirements development, systems engineering, design, integration, cost estimating, schedule tracking, test and evaluation, EVM, issue management, sustainment, and so on. The government program office, the prime contractor(s), and associated subcontractors should employ a compatible risk management process to facilitate the alignment of risk registers and transfer of data between parties.

The Request for Proposal (RFP) should address risk management by requiring that the offeror include in the proposal the nature of tasks, processes, and tools planned for collaborative government and industry risk management to ensure mutual understanding of risk efforts. The PM determines contract incentives and contract type to align government and contractor interests and risk management objectives.

The program’s risk profile is the dominant consideration in deciding which contract type to pursue.

Things to consider include firmness of the requirements and maturity of the technology required; the experience level of potential offerors; and the capacity of industry to absorb potential overruns and the business case for industry to do so. The type of contract, cost-plus or fixed-price, fundamentally will affect the roles and actions of the government and industry in managing risk.

Cost type contracts: Cost type contracts are best suited to situations in which the inherent technical risks are greater (typically during development). Consequently, these programs will need to allocate sufficient resources to manage emerging risks and should reevaluate sufficiency of funds during budget cycle reviews, before acquisition milestones, and with the award of follow-on contracts.

Government retains control in a cost type environment and should make (with the prime contractor)

8 DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs final decisions on risk mitigation plans. Although a contractor may have responsibility for managing a risk, the government still has ownership and responsibility for the efforts and outcomes.

Fixed-price contracts: Fixed-price development is most appropriate when the requirements are stable and expected to remain unchanged, where technical and technology risks are understood and minimal, and the contractor has demonstrated a capability to perform work of the type required. PMs and their contracting officers should reach an agreement with industry contractors during contract negotiations on how key risks must be mitigated, when progress will be measured, and any appropriate contract incentives and options. Although a contractor may have financial responsibility for mitigating a risk on a fixed-price contract, the government needs the product and bears the risk if the contractor fails to deliver it in a timely manner so the risk is never fully transferred to industry.

Appendix A includes a list of typical government and contractor responsibilities regarding risk management.

Pre-Materiel Development Decision Phase 2.2

This period is the best opportunity for the acquisition community to provide a balanced view to the users of what is realistically possible to achieve. Collaborative planning between the operational user and the technology/acquisition communities informs the translation of capability needs to initial requirements, and can guide technology investments or transition opportunities for candidate solutions. Early systems engineering provides trade space analysis for alternative candidate concepts.

Therefore, the requirements community and the project manager/organization should establish a close dialogue and relationship in this phase to account for the key risks in program planning.

Industry can also help. Development Planning (DP) organizations, working the concept prior to the Materiel Development Decision (MDD), can solicit ideas from industry through carefully crafted Requests for Information that seek insight into concepts, technologies, materials, and research investments. Industry feedback can help bound the realm of the possible.

Products of the pre-Materiel Development Decision (MDD) period include the formulation of the Initial Capabilities Document (ICD) and the Analysis of Alternatives (AoA) guidance for the conduct of the AoA during the MSA phase. The PM and acquisition community should participate in these activities and the initial development of acquisition approaches for the alternatives under consideration.

Materiel Solution Analysis Phase 2.3

In the MSA phase, the program conducts the analyses and other activities needed to finalize the concept of operations for the program product, refine the requirements, and conduct planning to support a decision on the acquisition strategy for the product. A key risk management activity during this phase is an engineering analysis of the ICD to better identify risks during the AoA. The program should evaluate requirements for technical feasibility, quantify gaps, and focus on contributing

9DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs technology components. Engineering analyses should focus on the affordability analysis, risk analysis, risk management planning, and trades among cost, schedule, and performance.

Acquisition sponsors should consider providing industry with draft technical requirements. Funded competitive concept definition studies (e.g., early design trade studies and operations research) can also inform decisions about requirements and are valuable to help refine and support requirements definition. Early industry feedback provides critical insight into the trade-offs among requirements, risks, and costs. Figure 2-2 displays the MSA phase activities.

Figure 2-2. Materiel Solution Analysis Phase Activities

The Service sponsor plans for an AoA to support the selection of a materiel solution. This planning includes both the AoA Guidance and an AoA Study plan. These guide the study team to assess cost, schedule, technical, and programmatic risk to inform the available trade space and cost-benefit analysis to shape affordable technical development. AoA focus areas should include uncertainty (or confidence level) associated with each alternative’s schedule estimate, proposed performance, and technical risks. The AoA study team should assess each of the following for realism relative to prior analyses and related systems:

• Interfaces and dependencies that involve other programs and the maturity and risks associated with the interfaces themselves (integration risk).

• Critical technologies required for each alternative: What is the present maturity of each?

What are the risks associated with bringing the critical technologies to the needed levels of maturity in a timely and cost-effective manner (technology risk)?

• Framing assumptions: Are these assumptions still valid? What are the risks and impacts of an incorrect assumption?

Following the selection of the preferred materiel solution, the PM should conduct an Alternative System Review (ASR) to support a dialogue between the end user and acquisition community, leading to a draft performance specification for the preferred materiel solution. The PM should also establish the program risk register to ensure the program has identified, analyzed, and established mitigation plans for all relevant risks.

By the end of the MSA phase, the program has focused on a single materiel solution and needs to plan for the next phase of activity. The maturity of the design and the nature of remaining risks will drive the decision about which phase will come next (i.e., Technology Maturation and Risk Reduction (TMRR), EMD, or Production and Deployment (P&D) phase). The program should address these risks in the RFP and program plans.

Materiel Solution Analysis

ICD

RFPDraft

CDD

Analysis of Alternatives and Engineering Analysis AoA

Study Plan Requirements Definition and Analysis

AMDD

ASR

Gov.

Req.

Rev.

AoA AoA

Senior Advisory Group (SAG) SAG

10 DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

2.3.1 Suggested Activities in the MSA Phase2 to Reduce Risk

Significant risk management activities of the MSA phase include the following:

• Establish an affordability goal and schedule and performance margins.

• Develop design concepts to assess the state of the possible and inform the requirements, the draft RFP and Request for Information (RFI), and source selection activities.

• Avoid constraining the design trade space (e.g., minimize the number of Key Performance Parameters (KPP) and Key System Attributes (KSA), per the Joint Capabilities Integration and Development System (JCIDS) Manual).

• Ensure the government and bidders have a complete and common understanding of the requirements

• Solicit industry feedback regarding the feasibility of requirements, unit costs, and maturity of technologies via industry days, meetings with prospective bidders, and RFIs.

• Hold a government-only requirements review to ensure the proper translation of the user requirements into the performance specification.

• Where appropriate, define the requirement in the performance specification for open systems architectures and interfaces, which can reduce the costs and time for changes or upgrades.

• Identify system (hardware and software) assurance risks early to ensure system requirements, design, and architecture will produce a secure system in operations.

• Ensure critical technologies are achievable. Risks should be manageable within schedule and resource constraints. Limit the number of critical technologies, as appropriate.

o Structure TMRR phase activities to validate performance during build, integration, and test to ensure requisite performance can be demonstrated by EMD.

o Ensure TMRR phase proposals include assessments of the maturity of proposed technologies. Validate with independent subject matter expert (SME) risk assessment.

o Collaborate with the S&T community to develop relevant technology, and request S&T dollars to mature key technologies.

• Focus the competitive prototyping strategy (if selected) on burning down the most critical technical risks (e.g., technology, engineering, and integration).

• Ensure the next phase RFP requires the contractors’ proposals to include:

o Contractor testing with defined success criteria before the start of government testing

2 MSA activities in the areas of requirements, technology, integration, test, and manufacturing may apply across multiple acquisition phases. Although they may not be repeated in the following sections, programs should consider the full range of activities when tailoring plans for a particular program and risk area.

11DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs o A requirement for contractors to identify problematic requirements as well as the cost and schedule associated with the requirements in their proposals to support the early maturation of the Capability Development Document (CDD) requirements.

• Ensure the TMRR phase RFP requires bidding contractors to identify risks and to provide an integration plan, an Integrated Master Schedule (IMS) through prototype delivery, and drawings/models so the government can assess (1) the contractors’ understanding of the technical risks and (2) the required planning to execute the plans.

o Develop a realistic program schedule, with appropriate phasing, which reflects consideration of relevant historical schedules as opposed to relying solely on an externally imposed timeline.

o Be event-driven versus schedule-driven to ensure risks are mitigated before the program proceeds to the next phase; ensure the schedule reflects an acceptable level of concurrency.

• Establish communication: horizontal, across Integrated Product Teams (IPT) and joint risk boards; and vertical, up through management on both the government and contractor sides.

Continue through all life cycle phases.

• Engage senior leadership from within the acquiring command, sponsor, and user community to manage program risks.

o Build an external senior leader stakeholder group and working groups.

o Ensure stakeholders understand the basis for the technical requirements so they feel ownership for appropriate risk reduction activities.

Technology Maturation and Risk Reduction Phase 2.4

If a TMRR phase is necessary, it should focus on reducing risks in technology, engineering, integration, and life cycle cost to the point that the Milestone Decision Authority (MDA) can make an EMD decision with confidence that the cost and schedule objectives carry understood and manageable risk. If the requirements community has clear and stable requirements and the supporting technology is mature, it may be possible to skip this phase and go directly to EMD or beyond.

Key risk areas include system performance and affordability. The PM decides what risk reduction activities to conduct in the TMRR phase but should prioritize starting with elements that represent the highest risk that can be reduced during this period of lower financial commitment. The PM should consider including special contract incentives for the high-risk areas. Typically, these activities include risk-reduction prototyping (which may be competitive) of the system, critical subsystems, technology, subcomponent, or component level. Prototyping of immature technologies can help inform decisions on how and whether to proceed. Another TMRR phase risk reduction activity is to identify and assess the materials and manufacturing processes the program will require.

Figure 2-3 displays the TMRR phase activities, including the following Systems Engineering Technical Reviews (SETR) to assess and manage risk: System Requirements Review (SRR), System

12 DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

Functional Review (SFR), and Preliminary Design Review (PDR). Throughout the TMRR phase, the program team should conduct a rigorous assessment of technical risk, develop risk mitigation options, and execute and monitor risk mitigation plans.

Figure 2-3. Technology Maturation and Risk Reduction Phase Activities

2.4.1 Suggested Activities and Practices in the TMRR Phase3 to Reduce Risk

Significant risk management activities of the TMRR phase include the following:

• Assess TMRR risks mitigation effectiveness, and evidence that the program has demonstrated critical technologies in a relevant environment and end-item design context.

• Develop an EMD schedule that includes time for integration, interdependency linkages, and mitigation of manufacturing risks.

• Conduct system or subsystem risk reduction prototyping validation and competition.

• Assess the preliminary design and allocated baseline to identify problematic requirements and risks to meeting operational requirements, technical achievability, and cost/affordability targets. Ensure derived requirements do not contribute to requirements creep.

• Conduct systems engineering trade-off and preliminary design activities to support the assessment of final requirements in the CDD.

• Incorporate decisions from Configuration Steering Board (CSB) meetings and/or knowledge point review (includes requirements and intelligence communities).

• Track program interdependencies, interfaces, and associated memorandums of agreement (MOA) in periodic meetings with external programs and associated contractors/stakeholders.

• Verify validity of program framing assumptions.

• Plan for contingencies and technical risk mitigation activities by establishing reasoned cost, schedule, and performance margins.

o Ensure risk mitigation plans are reflected in the Integrated Master Plan (IMP), IMS, Technical Performance Measures (TPM), and the EVM baseline. This may or may not require a change to the contractor work packages or resources. (Continue in subsequent phases.)

3 These practices may apply to the EMD phase as well. Programs should consider the range of activities when tailoring their risk mitigation plans.

Req. Definition & Analysis Functional Analysis Functional Allocation Detail Design

Draft

CDD

SE Trade-off Analysis

Risk Reduction and Competitive Prototyping Efforts

Technology Maturation & Risk Reduction

IBR

A

SRR SFRTRA TRAPDR

BReq.

Dec.

Dev

RFP

Dec.

PDR

or

13DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs o Identify resourced off-ramps for any critical technologies in the IMS.

o Avoid allowing the urgency of the schedule to outweigh good engineering and management.

• Conduct a government Technology Readiness Assessment (TRA) and risk assessment early in the TMRR phase. Ensure prototyping activities are relevant to the planned end item design and include plans to demonstrate technologies that present uncertainty.

o Identify applicable commercial technologies and develop an integration plan.

o Consider directed options (directed subcontractors) as opposed to industry teaming.

o Enable early evaluation of risks by planning an effective developmental test and evaluation program with adequate test articles and schedule duration for regression testing.

• Conduct a government TRA before MS B to identify and assess critical technology elements in the contractor’s EMD proposal.

• Conduct schedule risk analyses (SRA) on a regular basis to evaluate the likelihood to achieve the planned schedule.

• For trade studies affecting KPP/KSAs, develop a decision hierarchy to promptly identify and mitigate technical risks and their impact on cost, schedule, and performance.

• Consider S&T investments to support EMD and beyond.

• Develop MOAs with all external interdependent programs.

Engineering and Manufacturing Development Phase 2.5

The decision to enter EMD should be made when the design is mature, the requirements are stable, and the risks are acceptable. By entering this phase, a program commits to a product. It initiates the Department’s efforts for full-scale development and testing of a product to support verification of all operational and derived requirements so the program can begin production and deployment.

During the EMD phase, the program manages the remaining risk, builds and tests production-representative prototypes or first articles to verify compliance with requirements, and prepares for production and fielding. It includes the establishment of the product baseline for all configuration items.

Figure 2-4 displays the EMD phase activities. The program should conduct a Critical Design Review (CDR), a System Verification Review (SVR), a Functional Configuration Audit (FCA), and a Production Readiness Review (PRR) as part of its ongoing systems engineering and risk management efforts to assess and manage risk. These SETRs are technical milestones to assess the product and processes to ensure the system can perform as desired and proceed into the next phase within cost and schedule constraints at an acceptable level of risk.

14 DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

Figure 2-4. Engineering and Manufacturing Development Phase Activities

The PM should focus the risk management activities on the transition from development to production. The program should consider conducting a manufacturing readiness assessment before Low-Rate Initial Production (LRIP) and again before Full-Rate Production (FRP) to identify risks related to critical manufacturing processes and product characteristics. Examples of specific risk areas include requirements/design stability, integration and interdependency risks, and manufacturing/supply chain quality.

2.5.1 Suggested Activities in the EMD Phase to Reduce Risk

Additional risk management activities to reduce risk exposure in this phase include the following:

• Continue knowledge point reviews, CSB meetings, and assessment of framing assumptions as in the TMRR phase. When not making a change to KPPs could jeopardize a program’s utility or affordability, coordinate with the Joint Requirements Oversight Council.

• Update requirements trace and risk assessment for the draft Capability Production Document

(CPD).

• Conduct early risk-focused developmental testing with adequate time for necessary regression tests.

• Work with the operational test and evaluation community for early participation, requirements trace, and assessment.

• Require contractor testing with predefined success criteria to facilitate resolving integration activities and failure modes before the start of government testing.

• Establish and manage size, weight, power, and cooling (SWAP-C) performance and R&M allocations for all subsystems.

• Align logistics analysis, training, and support systems with system development.

• Plan technology refresh cycles to be implemented in the P&D and O&S phases to address technology obsolescence risks.

Production and Deployment Phase 2.6

The purpose of the P&D phase is to produce and deliver requirements-compliant products to receiving military organizations. The design must be stable enough to commit to production prior to entering this phase.

Final

CPDB C

CDR SVR/

FCA PRRTRRIBR

Detail Design Initial Fabrication

Engineering & Manufacturing Development (EMD)

15DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

Figure 2-5 displays the P&D phase activities. Specific actions include implementing mitigation plans for achieving Initial Operational Capability (IOC) and Full Operational Capability (FOC), which entails updating risk mitigation plans to address production and sustainment risks.

Figure 2-5. Production and Deployment Phase Activities

Following Milestone C, the program continues to mitigate risks to successful completion of Initial Operational Test and Evaluation (IOT&E) and the Physical Configuration Audit (PCA). The PCA verifies and validates that the product built is compliant with the design and meets the CPD. It also identifies technical risks for fielding and sustainment.

2.6.1 Suggested Activities in the P&D Phase to Reduce Risk

Additional activities to reduce risk exposure in the P&D phase include the following:

• Conduct a thorough PCA to verify production does not introduce new risks.

• Address risk associated with new requirements, follow-on increments, or deferred activities.

• Work with the Defense Contract Management Agency (DCMA) to assess production schedule risks.

o Identify and assess delivery schedule dependencies with external programs/users.

• Identify sustaining engineering needs and fund as appropriate.

Operations and Support Phase 2.7

In the Operations and Support (O&S) phase, the risk activities include monitoring in-service usage, problem reports, parts availability/obsolescence, engineering modifications, technology insertions, and operational hazard risks. The Service support organizations often work with the program offices.

During this phase, programs should plan for and establish In-Service Reviews (ISR). The ISR is a multi-disciplined assessment to characterize the in-service health of the deployed system and the enabling system elements (training, user manuals, documentation, etc.). Risk management activities in the course of the ISR include risk assessment of operational hazards, product baseline integrity, supply chain status, determination of acceptable operational hazard risk, and in-service usage/support risk.

C

IOT&EOTRR

FRP

PCAIBR

Production

Production & Deployment

16 DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

3 RISK AND ISSUE MANAGEMENT

Risk and issue management are closely related and use similar processes. All defense programs encounter risks and issues and must anticipate and address them on a continuing basis.

Risks are characterized by probability of occurrence and consequence. Through risk management, programs apply resources to lessen the likelihood of a future event occurring or the consequence should it occur. As risks increase in probability, programs should anticipate that the events will occur (i.e., become issues) and should put plans in place early to mitigate the consequences.

An issue differs from a risk in that its occurrence is certain, not probabilistic. An issue is characterized by its consequence, and issue management applies resources to address and reduce the potential negative consequences associated with a past, present, or certain future event. Issues may occur when a previously identified risk is realized, or they may occur without prior recognition of a risk. In addition, issues may spawn new risks.

Figure 3-1 depicts a suggested five-step management process that may be applied to a discrete risk or issue. The steps are broadly applicable to multiple phases in the program life cycle, but the details of particular actions will vary depending on program phase. This process for managing individual risks and issues operates within a broader framework in which consideration of risk shapes the basic program structure and content. Sections 3.1–3.5 further discuss risks. Selected aspects of the discussion may also apply to issues, discussed specifically in Section 3.6.

Figure 3-1. Risk and Issue Management Process Overview

Process Planning What are the program’s risk and issue management processes?

Identification What has, can, or will go wrong?

Analysis What is the likelihood of the risk and the consequence of the risk or issue?

Mitigation / Correction

What, if anything, will be done about the risk or issue?

Monitoring How has the risk or issue changed?

Communication and Feedback

17DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs

3 Risk and Issue Management

3.1 Risk Process Planning

Risk process planning consists of the program’s activities to develop, implement, and document steps the program will take to mitigate individual risks. The SEP should summarize the process. If a program develops a PRP (Program Risk Process) document that describes the process in more detail, the program can refer to the PRP in the SEP. For example, the PRP should describe the program’s risk management expectations, risk management organization (e.g., Risk Management Boards (RMB), frequency of meetings and members), ground rules and assumptions, candidate risk categories, use of risk management tools, and training of program personnel. The PRP should mention how often the document will be reviewed and updated.

Figure 3-2 summarizes the aspects of risk process planning. The planning should outline each of the risk management steps described in the succeeding sections.

Figure 3-2. Risk Process Planning

The process for managing individual risks applies to all program phases, but as uncertainties and risks change with each program phase, the plans for specific actions to manage risk will change as well. Early in the acquisition cycle, the program focuses on shaping the program structure and content needed to produce a desired capability. In the TMRR phase, the focus shifts to gaining a thorough understanding of risks and to mitigating the actual risks in the context of designing the specific product to be delivered. As the program approaches entry to the production phase, the emphasis is on whether the product design meets requirements and is stable, producible, and affordable. The focus on the central risks in each phase is on achieving the necessary levels of assurance to proceed.

Appendix A provides additional information on establishing a risk planning process.

When establishing risk processes and procedures:

• Assign roles, responsibilities, and…