Risk_Management_Guide.pdf

i

RISK

MANAGEMENT

GUIDE FOR

DOD ACQUISITION

Fifth Edition (Version 2.0)

Department of Defense Defense Acquisition University

June 2003

Report Documentation Page Form Approved OMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

1. REPORT DATE

00 JUN 2003

2. REPORT TYPE

N/A

3. DATES COVERED

4. TITLE AND SUBTITLE

Risk Management Guide for DOD Acquisition, Fifth Edition, (Version 2.0)

5a. CONTRACT NUMBER

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

Department of Defense, Defense Acquisition University, Ft. Belvoir, 22060-5565

8. PERFORMING ORGANIZATION

REPORT NUMBER

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR’S REPORT

NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release, distribution unlimited

13. SUPPLEMENTARY NOTES

14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF

ABSTRACT

UU

18. NUMBER

OF PAGES

19a. NAME OF

RESPONSIBLE PERSON

a. REPORT unclassified

b. ABSTRACT unclassified

c. THIS PAGE unclassified

Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 ii

PUBLISHED BY THE

DEFENSE ACQUISITION UNIVERSITY PRESS

FORT BELVOIR, VIRGINIA 22060-5565

Please e-mail comments or recommended changes to:

Bill.Bahnmaier@dau.mil

For sale by the U.S. Superintendent of Documents, Government Printing Office

Internet: bookstore.gpo.gov Phone: (202) 512-1800 Fax: (202) 512-2250 Mail Stop: SSOP, Washington, DC 20402-0001 iii

OFFICE OF THE UNDER SECRETARY OF DEFENSE

3000 DEFENSE PENTAGON

WASHINGTON, DC 20301-3000

RISK

MANAGEMENT

GUIDE

Acquisition excellence has changed the way the Department of Defense (DoD) designs, develops, manufactures, and supports systems. Our technical, business, and management approach for acquiring and operating systems has, and continues to, evolve. For example, we no longer can rely on military specifications and standards to define and control how our developers design, build, and support our new systems. Today we use commercial hardware and software, promote open systems architecture, and encourage streamlining processes, just to name a few of the initiatives that affect the way we do business. At the same time, the Office of the Secretary of Defense (OSD) has reduced the level of oversight and review of programs and manufacturers’ plants.

While the new acquisition model gives government program managers and their contractors broader control and more options than they have enjoyed in the past, it also exposes them to new risks. OSD recognizes that risk is inherent in any acquisition program and considers it essential that program managers take appropriate steps to manage and control risks.

This document is a product of a joint effort by the Under Secretary of Defense (Acquisition, Technology and Logistics (USD (AT&L)) staff and the Defense Acquisition University. It is based on the material developed by the DoD Risk Management Working Group. Material in this Guide is also reflected in the Risk Management Focus Area of the Program Management Community of Practice (PMCOP) (http://www.pmcop.dau.mil), and in the Defense Acquisition Deskbook, which can be accessed via the AT&L Knowledge Sharing System (AKSS) Website (http://deskbook.dau.mil/jsp/default.jsp).

Frank J. Anderson, Jr.

President Defense Acquisition University iv

PREFACE

In 1996, the USD (AT&L) established a Risk Management Working Group composed of members of the Office of the Secretary of Defense (OSD) staff, representatives of the Military Services, and members of other DoD agencies involved in systems acquisition. This group reviewed pertinent DoD directives (DoDD) and regulations, examined how the Services managed risk, studied various ex-amples of risk management by industry, and looked at DoD training and education activity in risk management. Other sources of information were the Software Engineering Institute Risk Initiative, the Open Systems Initiative, and the safety and cost estimating communities. The findings and results of the Working Group investigation were presented to the USD (AT&L) and are summarized below:

Working Group members then wrote the risk management portions of the Defense Acquisition Desk-book. The Defense Acquisition Deskbook (sometimes referred to as the “Legacy” Deskbook) is ac-cessible from the AT&L Knowledge Sharing System (AKSS) Website (http://deskbook.dau.mil/ jsp/default.jsp).

Industries

• Focus of efforts is to get a product to market at a competitive price.

• Industry has have either a structured or informal Risk Management process.

• Evolutionary approaches help avoid or minimize risk.

• Most approaches employ risk avoidance, early planning, continuous assessment, and problem-solving techniques.

• Structured approaches, when they exist, are similar to DoD’s approach to Risk Management.

The Working Group concluded that industry has no magic formula for Risk Management.

The Military Services

• The Services differ in their approaches to Risk Management.

• Each approach has its strengths but no one approach is comprehensive.

• Consolidation of the strengths of each approach could foster better Risk Management in DoD.

The Working Group recommended that the Defense Acquisition Deskbook contain a set of guidelines for sound risk management practices, and further, that it contain a set of risk management definitions that are comprehensive and useful by all the Components.

DoD Policy*

• The risk management policy contained in DoDD 5000.1 is not comprehensive.

The Working Group recommended that DoDD 5000.1 be amended to include a more comprehensive set of risk management policies that focuses on:

• The relationship between the Cost As an Independent Variable (CAIV) concept and Risk Management.

• Requirement that risk management be prospective (forward looking).

• Establishment of risk management as a primary management technique to be used by Program Managers (PMs).

*Note: The DoD 5000 policy documents referred to in the 1996 Report have since been superseded by a new set of DoD 5000 policy and guidance documents issued in 2002–2003 time frame.

v

The recommendations of the Risk Management Working Group have been fully implemented over the period 1996-2003. The Risk Management part of the Defense Acquisition Deskbook and material in the Risk Focus Area of the Program Management Community of Practice (PMCoP) (http:// www.pmcop.dau.mil) form the basis for this Guide. The goal of the Risk Management Guide is to provide acquisition professionals and program management offices with a practical reference for deal-ing with system acquisition risks. It has also been designed to be used as an aid in DAU course offerings.

This Guide reflects the efforts of many people. Mr. Mark Schaeffer, former Deputy Director, Systems Engineering, who chaired the initial Risk Management Working Group, and Mr. Mike Zsak and Mr.

Tom Parry, formerly from the AT&L Systems Engineering Support Office, were the original driving forces behind the risk management initiative. LtCol John Driessnack, USAF, from the DAU/DSMC faculty; Mr. Greg Caruth, Ms. Debbie Gonzalez, and Ms. Frances Battle from the DAU Press; and Ms. Patricia Bartlett from Bartlett Communications guided the composition of the Guide. Assistance was also provided by Mr. Jeff Turner of the DAU Publications Distribution Center. Special recogni-tion goes to the Institute for Defense Analyses team composed of Mr. Louis Simpleman, Mr. Ken Evans, Mr. Jim Lloyd, Mr. Gerald Pike, and Mr. Richard Roemer, who compiled the data and wrote major portions of the text. Also special thanks to Ms. Margaret Adcock for her detailed comments and support, and to Dr. Edmund Conrow for his suggestions and recommendations that have vastly improved the Guide.

Charles B. Cochrane William W. Bahnmaier Director Editor DAU Center for Program Management

DoD Procedures

• Risk Management procedures in DoD 5000.2-R (Note: Later changed to Interim Defense Acquisi-tion Guidebook) are inadequate to fully implement the risk management policy contained in DoDD 5000.1.

Procedures are lacking regarding:

– Scope of Risk Management

– Purpose of Risk Management

– Role of Milestone Decision Authorities

– Risk Management’s support of CAIV

– Risk assessment during early acquisition phases.

• Some key procedures may have been lost in transition the DoD 5000.2-R, and need to be ex-panded upon in the Defense Acquisition Deskbook.

DoD Risk Management Training

• Risk management training for the DoD Acquisition Corps needs to be updated and expanded, and Integrated Product Team (IPT) and Overarching IPT (OIPT) personnel need to be educated on the new and expanding role of risk management in DoD systems acquisition.

• Risk Management knowledge level needs improvement.

• Education is a key to obtaining the support of OIPTs and PMs. The Defense Acquisition Univer-sity (DAU) needs to include Risk Management training in all functional courses and develop a dedicated risk management course for acquisition corps personnel.

vi

CONTENTS

Chapter 1 INTRODUCTION

1.1 Purpose and Scope

1.2 Organization of the Guide

1.3 Approach to Risk Management

1.4 DoD Risk Management Policies and Procedures

Chapter 2 RISK AND RISK MANAGEMENT

2.1 Introduction

2.2 Overview

2.3 Risk Management Structure and Definitions

2.4 Risk Discussion

2.4.1 Characteristics of Acquisition Risk

2.4.2 Program Products, Processes, Risk Areas, and Risk Events

2.5 Risk Planning

2.5.1 Purpose of Risk Plans

2.5.2 Risk Planning Process

2.6 Risk Assessment

2.6.1 Purpose of Risk Assessments

2.6.2 Risk Assessment Process

2.6.3 Timing of Risk Assessments

2.6.4 Conducting Risk Assessments

2.7 Risk Handling

2.7.1 Purpose of Risk Handling

2.7.2 Risk-Handling Process

2.8 Risk Monitoring

2.9 Risk Documentation

Chapter 3 RISK MANAGEMENT AND THE DOD ACQUISITION PROCESS

3.1 Introduction

3.2 Overview

3.3 DoD Acquisition Process

3.4 Characteristics of the Acquisition Process

3.4.1 Integrated Product and Process Development (IPPD)

3.4.2 Continuous Risk Management

3.4.3 Program Stability

3.4.4 Reduction of Life-Cycle Costs

vii

3.4.5 Event-Oriented Management

3.4.6 Modeling and Simulation

3.5 Risk Management Activities during Acquisition Phases

3.5.1 Concept Refinement (CR) and Technology Development (TD) Phases

3.5.2 Subsequent Phases

3.6 Risk Management and Milestone Decisions

3.7 Risk Management and the Acquisition Strategy

3.8 Risk Management and CAIV

Chapter 4 RISK MANAGEMENT AND PROGRAM MANAGEMENT

4.1 Introduction

4.2 Overview

4.3 Program Manager and Risk Management

4.3.1 Risk Management Is a Program Management Tool

4.3.2 Risk Management Is a Formal Process

4.3.3 Risk Management Is Forward-Looking

4.3.4 Risk Management Is Integral to Integrated Product

and Process Development (IPPD)

4.4 Risk Management Organization in the PMO

4.4.1 Risk Management Organizational Structure

4.4.2 Risk Management Responsibilities

4.5 Contractor Risk Management

4.5.1 Contractor View of Risk

4.5.2 Government/Contractor Relationship

4.6 Risk Management and the Contractual Process

4.6.1 Risk Management: Pre-Contract Award

4.6.2 Early Industry Involvement: Industrial Capabilities Review

4.6.3 Developing the Request for Proposal

4.6.4 The Offeror’s Proposal

4.6.5 Basis for Selection

4.6.6 Source Selection

4.7 Risk Management: Post-Contract Award

4.8 Risk Management Reporting and Information System

4.9 Risk Management Training

Chapter 5 RISK MANAGEMENT TECHNIQUES

5.1 Introduction

5.2 Overview

5.3 Risk Planning Techniques

5.3.1 Description

5.3.2 Procedures

viii

5.4 Risk Assessment Techniques

5.4.1 Product (WBS) Risk Assessment

5.4.2 Process (DoD 4245.7-M) Risk Assessment

5.4.3 Program Documentation Evaluation Risk Identification

5.4.4 Threat and Requirements Risk Assessment

5.4.5 Cost Risk Assessment

5.4.6 Quantified Schedule Risk Assessment

5.4.7 Expert Interviews

5.4.8 Analogy Comparison/Lessons-Learned Studies

5.5 Risk Prioritization

5.5.1 Description

5.5.2 Procedures

5.6 Risk-Handling Techniques

5.6.1 General

5.6.2 Risk Control

5.6.3 Risk Avoidance

5.6.4 Risk Assumption

5.6.5 Risk Transfer

5.7 Risk Monitoring

5.7.1 General

5.7.2 Earned Value Management

5.7.3 Technical Performance Measurement

5.7.4 Integrated Planning and Scheduling

5.7.5 Watch List

5.7.6 Reports

5.7.7 Management Indicator System

5.8 Risk Management Information Systems and Documentation

5.8.1 Description

5.8.2 Risk Management Reports

5.9 Software Risk Management Methodologies

5.9.1 Software Risk Evaluation (SRE)

5.9.2 Boehm’s Software Risk Management Method

5.9.3 Best Practices Initiative Risk Management Method

APPENDIX A –

DOD RISK MANAGEMENT POLICIES AND PROCEDURES ................................... A-1

DoD Directive 5000.1. The Defense Acquisition System, 12 May 2003 ........................... A-1

DoD Instruction 5000.2. Operation of the Defense Acquisition System, 12 May 2003 ..... A-1

Interim Defense Acquisition Guidebook (IDAG), 30 October 2002 .................................. A-4

DoD Directive 5000.4. OSD Cost Analysis Improvement Group (CAIG), 24 November 1992 ..................................................................................................... A-12 ix

DoD 5000.4-M. Cost Analysis Guidance and Procedures, December 1992 .................... A-13

APPENDIX B –

GENERIC RISK MANAGEMENT PLAN ........................................................................ B-1

Sample Risk Management Plan .......................................................................................... B-1

Preface ............................................................................................................................... B-1

Sample Format for Risk Management Plan ........................................................................ B-2

Sample Risk Management Plan for the XYZ Program (ACAT I, II) .................................. B-4

1.0 Introduction ......................................................................................................... B-4

1.1 Purpose ............................................................................................................... B-4

1.2 Program Summary .............................................................................................. B-4

1.2.1 System Description ................................................................................. B-5

1.2.2 Acquisition Strategy ................................................................................ B-5

1.2.3 Program Management Approach ............................................................. B-5

1.3 Definitions ........................................................................................................... B-5

1.3.1 Risk ......................................................................................................... B-5

1.3.2 Risk Event ............................................................................................... B-5

1.3.3 Technical Risk ......................................................................................... B-6

1.3.4 Cost Risk ................................................................................................. B-6

1.3.5 Schedule Risk ......................................................................................... B-6

1.3.6 Risk Ratings ............................................................................................ B-6

1.3.7 Independent Risk Assessor ...................................................................... B-6

1.3.8 Templates and Best Practices .................................................................. B-6

1.3.9 Metrics .................................................................................................... B-7

1.3.10 Critical Program Attributes ...................................................................... B-7

2.0 Risk Management Approach ............................................................................... B-7

2.1 General Approach and Status .............................................................................. B-7

2.2 Risk Management Strategy ................................................................................. B-8

2.3 Organization ........................................................................................................ B-8

2.3.1 Risk Management Coordinator ............................................................... B-8

2.3.2 Program Level Integrated Product Team (PLIPT) ................................... B-9

2.3.3 PIPTs ...................................................................................................... B-9

2.3.4 XYZ Independent Risk Assessors ........................................................... B-9

2.3.5 Other Risk Assessment Responsibilities ................................................ B-10

2.3.6 User Participation .................................................................................. B-10

2.3.7 Risk Training ......................................................................................... B-10

3.0 Risk Management Process and Procedures ....................................................... B-10

3.1 Overview .......................................................................................................... B-10

3.2 Risk Planning .................................................................................................... B-11 x

3.2.1 Process .................................................................................................. B-11

3.2.2 Procedures ............................................................................................. B-11

3.3 Risk Assessment ................................................................................................ B-12

3.3.1 Process .................................................................................................. B-12

3.3.2 Procedures ............................................................................................. B-13

3.4 Risk Handling ................................................................................................... B-17

3.4.1 Process .................................................................................................. B-17

3.4.2 Procedures ............................................................................................. B-18

3.5 Risk Monitoring ................................................................................................ B-18

3.5.1 Process .................................................................................................. B-18

3.5.2 Procedures ............................................................................................. B-18

4.0 Risk Management Information System (RMIS) and Documentation ............... B-19

4.1 Risk Management Information System ............................................................. B-19

4.2 Risk Documentation .......................................................................................... B-20

4.2.1 Risk Assessment Documentation .......................................................... B-20

4.2.2 Risk-Handling Documentation .............................................................. B-20

4.2.3 Risk Monitoring Documentation ........................................................... B-20

4.3 Reports .............................................................................................................. B-20

4.3.1 Standard Reports ................................................................................... B-20

4.3.2 Ad Hoc Reports ..................................................................................... B-20

Annex A to XYZ Risk Management Plan – Critical Program Attributes ........................................................................... B-21

Annex B to XYZ Risk Management Plan – Program Risk Reduction Schedule ............................................................... B-22

Annex C to XYZ Risk Management Plan – Program Metric Examples ............................................................................ B-23

Annex D to XYZ Risk Management Plan – Management Information System and Documentation ................................. B-25

1.0 Description ...................................................................................... B-25

2.0 Risk Management Reports – XYZ Program.................................... B-25

2.1 Risk Information Form ............................................................ B-25

2.2 Risk Assessment Report .......................................................... B-26

2.3 Risk Handling Documentation ................................................ B-26

2.4 Risk Monitoring Documentation ............................................. B-26

3.0 Database Management System (DBMS) ......................................... B-26

Sample Risk Management Plan for the ABC Program (ACAT III, IV) ............................ B-31

1.0 Introduction ....................................................................................................... B-31

1.1 Purpose .................................................................................................... B-31

2.0 Program Summary ............................................................................................ B-31 xi

2.1 Description ............................................................................................... B-31

2.2 Acquisition Strategy ................................................................................. B-32

2.3 Program Management Approach .............................................................. B-32

3.0 Risk-Related Definitions ................................................................................... B-32

3.1 Technical Risk .......................................................................................... B-32

3.2 Cost Risk .................................................................................................. B-32

3.3 Risk Ratings ............................................................................................. B-32

4.0 Risk Management Status and Strategy .............................................................. B-33

4.1 Risk Management Status .......................................................................... B-33

4.2 Risk Management Strategy ...................................................................... B-33

5.0 Risk Management Organization ........................................................................ B-33

5.1 Program Office ......................................................................................... B-33

6.0 Risk Management Structures and Procedures ................................................... B-34

6.1 Risk Planning ........................................................................................... B-35

6.2 Risk Assessment ....................................................................................... B-35

6.2.1 Risk Identification ........................................................................ B-36

6.2.2 Risk Analysis ............................................................................... B-37

6.2.3 Risk Rating .................................................................................. B-39

6.2.4 Risk Prioritization ......................................................................... B-40

6.3 Risk Handling .......................................................................................... B-40

6.4 Risk Monitoring ....................................................................................... B-41

6.5 Risk Management Information System (RMIS), Documentation, and Reports ............................................................... B-42

Annex A to ABC Risk Management Plan – Critical Program Attributes .......... B-43

Annex B to ABC Risk Management Plan – Management Information System and Documentation ................................. B-44

1.0 Description ...................................................................................... B-44

2.0 Risk Management Forms and Reports ............................................. B-44

2.1 Risk Information Form ............................................................ B-44

2.2 Risk Monitoring Documentation ............................................. B-44

2.3 PIPT Risk Summary Report .................................................... B-44

APPENDIX C –

GLOSSARY ........................................................................................................................... C-1

APPENDIX D –

QUANTIFYING EXPERT JUDGMENT .......................................................................... D-1

APPENDIX E –

BIBLIOGRAPHY ................................................................................................................. E-1 xii

FIGURES

2-1. Risk Management Structure

2-2. Critical Process Areas and Templates

2-3. A Risk Management Plan Outline/Format

2-4. Risk Assessment

2-5. Example of a WBS Dependent Evaluation Structure

2-6. Overall Risk Rating (Example)

4-1. Decentralized Risk Management Organization

5-1. Risk Planning Technique Input and Output

5-2. Sample Format for Risk Management Plan

5-3. Product (WBS) Risk Assessment Technique Input and Output

5-4. Process (DoD 4245.7-M) Risk Assessment Technique Input and Output

5-5. Plan Evaluation Technique Input and Output

5-6. Concept Refinement (CR) and Technology Development (TD) Phases

Correlation of Selected Documents (Example)

5-7. Threat and Requirement Risk Assessment Technique Input and Output

5-8. Cost Risk Assessment Top-Level Diagram

5-9. Schedule Risk Assessment Technique Input and Output

5-10. Expert Interview Technique Input and Output

5-11. Analogy Comparison/Lessons-Learned Studies Top-Level Diagram

5-12. Risk Prioritization Technique Input and Output

5-13. Risk Aggregation Technique Input and Output

5-14. List of Aggregated Risks

5-15. Example Showing Detailed List of Top-Level Risk Information

5-16. Example of More Complex Combination of Risk Level and Scheduled Tasks

5-17. Conceptual Risk Management and Reporting System

B-1. Risk Management and the Acquisition Process ....................................................... B-7

B-2. XYZ Risk Management Organization ..................................................................... B-9

B-3. Risk Management Structure ................................................................................... B-10

B-4. Risk Assessment Process ....................................................................................... B-16

B-5. XYZ Program Risk Handling Plan Schedule (Example) ....................................... B-22

B-6. Conceptual Risk Management and Reporting System ........................................... B-25

B-7. Risk Information Form .......................................................................................... B-28

B-8. Risk Tracking Report Example .............................................................................. B-29

B-9. ABC Risk Management Organization ................................................................... B-33

B-10. Risk Assessment Process ....................................................................................... B-39

B-11. Example Risk Tracking Report .............................................................................. B-46 xiii

TABLES

2-1. Risk Assessment Approaches

2-2. Probability/Likelihood Criteria (Example)

2-3. Consequences/Impacts Criteria (Example)

2-4. Overall Risk Rating Criteria (Example)

2-5. Risk Ratings (Example)

4-1. Notional Description of Risk Management Responsibilities

4-2. Significant Risks by Critical Risk Areas

4-3. Risk Management Reference Documents

5-1. Critical Risk Areas and Example Elements

5-2. Examples of Demonstration Events

5-3. Watch List Example

5-4. Examples of Product-Related Metrics

5-5. Examples of Process Metrics

5-6. Examples of Cost and Schedule Metrics

5-7. Database Management System Elements

5-8. Software Risk Management Steps

5-9. Top 10 Software Risks

5-10. Best Practices Initiative Risk Management Method

5-11. Software Risk Grouping

B-1. Critical Program Attributes .................................................................................... B-21

B-2. Examples of Product-Related Metrics .................................................................... B-23

B-3. Examples of Process Metrics ................................................................................. B-23

B-4. Examples of Cost and Schedule Metrics ................................................................ B-24

B-5. DBMS Elements .................................................................................................... B-27

B-6. Watch List Example............................................................................................... B-30

B-7. Likelihood Levels .................................................................................................. B-38

B-8. Risk Consequence ................................................................................................. B-40

B-9. Critical Program Attributes .................................................................................... B-43

B-10. DBMS Elements .................................................................................................... B-45

B-11. Sample Watch List ................................................................................................. B-47

B-12. Example PIPT Risk Summary Report ................................................................... B-47

B-13. Examples of Process Metrics ................................................................................. B-48

B-14. Examples of Cost and Schedule Metrics ................................................................ B-48 xiv

INTRODUCTION

classroom instruction and as a reference book for practical applications. Most of the material in this Guide is derived from the Defense Acquisition Deskbook of the Acquisition, Technology, and Logistics (AT&L) Knowledge Sharing System (AKSS) and from the Risk Focus Area of the Program Management Community of Practice (PMCoP). Readers should refer to the PMCoP Website (http://www.pmcop.dau. mil) for any new risk management information that is dissemi-nated between publishing of updated Guide edi-tions or versions of editions.

1.2 ORGANIZATION OF THE GUIDE

The Risk Management Guide discusses risk and risk management, defines terms, and introduces basic risk management concepts (Chapter 2).

Chapter 3 examines risk management concepts relative to the DoD acquisition process. It illustrates how risk management is an integral part of program management, describes interac-tion with other acquisition processes, and iden-tifies and discusses the various types of acquisi-tion risks.

Chapter 4 discusses the implementation of a risk management program from the perspective of a PMO. This chapter focuses on practical appli-cation issues such as risk management program design options, PMO risk management organi-zations, and criteria for a risk management in-formation system (MIS).

Risk has always been a concern in the acquisi-tion of Department of Defense (DoD) systems.

The acquisition process itself is designed, to a large degree, to allow risks to be controlled from conception to delivery of a system. Unfortu-nately, in the past, some Program Managers (PMs) and decision makers have viewed risk as something to be avoided. Any program that had risk was subject to intense review and over-sight. This attitude has changed. DoD manag-ers recognize that risk is inherent in any pro-gram and that it is necessary to analyze future program events to identify potential risks and take measures to handle them.

Risk management is concerned with the out-come of future events, whose exact outcome is unknown, and with how to deal with these un-certainties, i.e., a range of possible outcomes.

In general, outcomes are categorized as favor-able or unfavorable, and risk management is the art and science of planning, assessing, and handling future events to ensure favorable out-comes. The alternative to risk management is crisis management, a resource-intensive process that is normally constrained by a restricted set of available options.

1.1 PURPOSE AND SCOPE

This Risk Management Guide is designed to provide acquisition professionals and program management offices (PMOs) with a practial ref-erence book for dealing with system acquisition risks. It is also intended to be useful as an aid in

Chapter 5, the final chapter, describes a number of techniques that address the aspects (phases) of risk management, i.e., planning, assessment, handling, and monitoring.

This Guide is a source of background informa-tion and provides a starting point for a risk man-agement program. None of the material is man-datory. PMs should tailor the approaches and techniques to fit their programs.

The Risk Management Guide also contains appendices that are intended to serve as refer-ence material and examples, and provide backup detail for some of the concepts pre-sented in the main portion of the Guide.

1.3 APPROACH TO RISK

MANAGEMENT

Based on the DoD model contained in the De-fense Acquisition Deskbook (described in Chap-ter 2), this Guide emphasizes a risk management approach that is disciplined, forward looking, and continuous.

In 1986, the Government Accounting Office (GAO), as part of an evaluation of DoD poli-cies and procedures for technical risk assess-ments, developed a set of criteria as an approach to good risk assessments. These criteria, with slight modification, apply to all aspects of risk management and are encompassed in the Guide’s approach. They are:

(1) Planned Procedures. Risk management is planned and systematic.

(2) Prospective Assessment. Potential future problems are considered, not just current problems.

(3) Attention to Technical Risk. There is explicit attention to technical risk.

(4) Documentation. All aspects of the risk man-agement program are recorded and data maintained.

(5) Continual Process. Risk assessments are made throughout the acquisition process;

handling activities are continually evaluated and changed if necessary; and critical risk areas are always monitored.

While these criteria are not solely sufficient to determine the “health” of a program, they are important indicators of how well a risk management process is being implemented. A pro-active risk management process is a good start toward a successful program.

1.4 DOD RISK MANAGEMENT

POLICIES AND PROCEDURES

DoD policies and procedures that address risk management for acquisition programs are con-tained in five key DoD documents. DoD Direc-tive (DoDD) 5000.1 (The Defense Acquisition System) contains overall acquisition policy — with a strong basis in risk management. The policy on risk management is amplified further by the information in DoD Instruction (DoDI)

5000.2 (Operation of the Defense Acquisition System) and the Interim Defense Acquisition Guidebook (IDAG). These documents integrate risk management into the acquisition process, de-scribe the relationship between risk and various acquisition functions, and establish some reporting requirements. DoDD 5000.4 and DoD 5000.4-M address risk and cost analysis guidance as they apply to the Office of the Secretary of Defense. Appendix A is an extract of existing risk management policies and procedures from all of these documents.

The DoD 5000 series contains strong statements on risk management but requires elaboration to help the PM establish an effective risk manage-ment program. The information furnished in the

Risk Management section of the Defense Ac-quisition Deskbook and in the Risk Focus Area of the PMCoP supports and expands the con-tents of the DoD 5000 series. This Guide in turn is derived from and reflects those sources.

RISK AND

RISK MANAGEMENT

insight into risk areas, thereby allowing the de-velopment of effective handling strategies. The net result promotes executable programs.

Effective risk management requires involve-ment of the entire program team and also re-quires help from outside experts knowledge-able in critical risk areas (e.g., threat, technol-ogy, design, manufacturing, logistics, schedule, and cost). In addition, the risk management pro-cess should cover hardware, software, the hu-man element, and integration issues. Outside experts may include representatives from the user, laboratories, contract management, test, logistics, and sustainment communities, and industry. Users, essential participants in pro-gram trade analyses, should be part of the as-sessment process so that an acceptable balance among cost, schedule, performance, and risk can be reached. A close relationship between the Government and industry, and later with the selected contractor(s), promotes an understand-ing of program risks and assists in developing and executing the management efforts.

Successful risk management programs gen-erally have the following characteristics:

€ Feasible, stable, and well-understood user requirements and threat;

• A close relationship with user, industry, and other appropriate participants;

2.1 INTRODUCTION

This Chapter introduces the concepts of risk and risk management by explaining the DoD risk-related definitions and by identifying the char-acteristics of acquisition risks. It also presents and discusses a structured concept for risk management and its five subordinate processes.

2.2 OVERVIEW

The DoD risk management concept is based on the principles that risk management must be forward-looking, structured, informative, and continuous. The key to successful risk manage-ment is early planning and aggressive execu-tion. Good planning enables an organized, com-prehensive, and iterative approach for identi-fying and assessing the risk and handling op-tions necessary to refine a program acquisition strategy. To support these efforts, assessments should be performed as early as possible in the life cycle to ensure that critical technical, sched-ule, and cost risks are addressed with handling actions incorporated into program planning and budget projections.

PMs should update program risk assessments and tailor their management strategies accord-ingly. Early information gives them data that helps when writing a Request for Proposal and assists in Source Selection planning. As a pro-gram progresses, new information improves

• A planned and structured risk management process, integral to the acquisition process;

• An acquisition strategy consistent with risk level and risk-handling strategies;

• Continual reassessment of program and associated risks;

• A defined set of success criteria for all cost, schedule, and performance elements, e.g., Acquisition Program Baseline (APB) thresholds;

• Metrics to monitor effectiveness of risk-handling strategies;

• Effective Test and Evaluation Program; and

• Formal documentation.

PMs should follow the guidelines below to ensure that a management program possesses the above characteristics.

• Assess program risks, using a structured pro-cess, and develop strategies to manage these risks throughout each acquisition phase.

• Identify early and intensively manage those design parameters that critically affect cost, capability, or readiness.

• Use technology demonstrations/modeling/ simulation and aggressive prototyping to reduce risks.

• Use test and evaluation as a means of quantifying the results of the risk-handling process.

• Include industry and user participation in risk management.

• Use Developmental Test and Evaluation (DT&E) and early operational assessments when appropriate.

• Establish a series of “risk assessment re-views” to evaluate the effectiveness of risk handling against clearly defined success criteria.

• Establish the means and format to communi-cate risk information and to train participants in risk management.

• Prepare an assessment training package for members of the program office and others, as needed.

• Acquire approval of accepted risks at the appropriate decision level.

In general, management of software risk is the same as management of other types of risk and techniques that apply to hardware programs are equally applicable to software intensive programs. Nevertheless, some characteristics of software make this type of risk management dif-ferent, primarily because it is difficult to:

• Identify software risk.

• Estimate the time and resources required to develop new software, resulting in potential risks in cost and schedule.

• Test software completely because of the number of paths that can be followed in the logic of the software.

• Develop new programs because of the rapid changes in information technology and an ever-increasing demand for quality software personnel.

2.3 RISK MANAGEMENT

STRUCTURE AND DEFINITIONS

Although each risk management strategy depends upon the nature of the system being developed, research reveals that good strategies contain the same basic processes and structure shown in Figure 2-1. This structure is some-times also referred to as the Risk Management Process Model. The application of these pro-cesses vary with acquisition phases and the de-gree of system definition; all should be inte-grated into the program management function.

The elements of the structure are discussed in the following paragraphs of this Chapter; how-ever, in order to form a basis for discussion, the Defense Acquisition Deskbook definitions for the processes and elements of risk management include:

Risk is a measure of the potential inability to achieve overall program objectives within de-fined cost, schedule, and technical constraints and has two components: (1) the probability/ likelihood of failing to achieve a particular out-come, and (2) the consequences/impacts of fail-ing to achieve that outcome.

Figure 2-1. Risk Management Structure

Risk events, i.e., things that could go wrong for a program or system, are elements of an acquisi-tion program that should be assessed to deter-mine the level of risk. The events should be de-fined to a level that an individual can compre-hend the potential impact and its causes. For ex-ample, a potential risk event for a turbine engine could be turbine blade vibration. There could be a series of potential risk events that should be selected, examined, and assessed by subject-matter experts.

The relationship between the two components of risk — probability and consequence/impact

— is complex. To avoid obscuring the results of an assessment, the risk associated with an event should be characterized in terms of its two com-ponents. As part of the assessment there is also a need for backup documentation containing the supporting data and assessment rationale.

Risk management is the act or practice of deal-ing with risk. It includes planning for risk, as-sessing (identifying and analyzing) risk areas, developing risk-handling options, monitoring risks to determine how risks have changed, and documenting the overall risk management program.

Risk Management

Risk Assessment

Risk Planning

Risk Handling

Risk Monitoring

Risk Identification

Risk Analysis

Risk Documentation

Risk planning is the process of developing and documenting an organized, comprehensive, and interactive strategy and methods for identify-ing and tracking risk areas, developing risk-handling plans, performing continuous risk as-sessments to determine how risks have changed, and assigning adequate resources.

Risk assessment is the process of identifying and analyzing program areas and critical tech-nical process risks to increase the probability/ likelihood of meeting cost, schedule, and per-formance objectives. Risk identification is the process of examining the program areas and each critical technical process to identify and document the associated risk. Risk analysis is the process of examining each identified risk area or process to refine the description of the risk, isolating the cause, and determining the effects. It includes risk rating and prioritization in which risk events are defined in terms of their probability of occurrence, severity of conse-quence/impact, and relationship to other risk areas or processes.

Risk handling is the process that identifies, evaluates, selects, and implements options in order to set risk at acceptable levels given pro-gram constraints and objectives. This includes the specifics on what should be done, when it should be accomplished, who is responsible, and associated cost and schedule. The most ap-propriate strategy is selected from these han-dling options. For purposes of the Guide, risk handling is an all-encompassing term whereas risk mitigation is one subset of risk handling.

Risk monitoring is the process that systemati-cally tracks and evaluates the performance of risk-handling actions against established metrics throughout the acquisition process and develops further risk-handling options, as appropriate. It feeds information back into the other risk management activities of planning, assessment, and handling as shown in Figure

2-1. This feedback mechanism was first sug-gested by Dr. Edmund Conrow in his book Effective Risk Management: Some Keys to Success.

Risk documentation is recording, maintaining, and reporting assessments, handling analysis and plans, and monitoring results. It includes all plans, reports for the PM and decision authorities, and reporting forms that may be internal to the PMO.

2.4 RISK DISCUSSION

Implicit in the definition of risk is the concept that risks are future events , i.e., potential prob-lems, and that there is uncertainty associated with the program if these risk events occur.

Therefore, there is a need to determine, as much as possible, the probability of a risk event occurring and to estimate the consequence/ impact if it occurs. The combination of these two factors determines the level of risk. For example, an event with a low probability of occurring, yet with severe consequences/impacts, may be a can-didate for handling. Conversely, an event with a high probability of happening, but the conse-quences/impacts of which do not affect a program, may be acceptable and require no handling.

To reduce uncertainty and apply the definition of risk to acquisition programs, PMs must be familiar with the types of acquisition risks, un-derstand risk terminology, and know how to measure risk. These topics are addressed in the next several sections.

2.4.1 Characteristics of Acquisition Risk

Acquisition programs tend to have numerous, often interrelated, risks. They are not always obvious; relationships may be obscure; and they may exist at all program levels throughout the life of a program. Risks are in the PMO (program plans, etc.); in support provided by other Gov-ernment agencies; in threat assessment; and in prime contractor processes, engineering and manufacturing processes, and technology. The interrelationship among risk events may cause an increase in one because of the occurrence of another. For example, a slip in schedule for an early test event may adversely impact subse-quent tests, assuming a fixed period of test time is available.

Another important risk characteristic is the time period before a risk future event occurs; because time is critical in determining risk-handling options. If an event is imminent, the PMO must resort to crisis management. An event that is far enough in the future to allow management actions may be controllable. The goal is to avoid the need to revert to crisis management and problem solving by managing risk up front.

An event’s probability of occurrence and con-sequences/impacts may change as the develop-ment process proceeds and information be-comes available. Therefore, throughout the de-velopment phase, PMOs should reevaluate known risks on a periodic basis and examine the program for new risks.

2.4.2 Program Products, Processes, Risk Areas, and Risk Events

Program risk includes all risk events and their relationships to each other. It is a top-level as-sessment of impact to the program when all risk events at the lower levels of the program are considered. Program risk may be a roll-up of all low-level events; however, most likely, it is a subjective evaluation of the known risks by the PMO, based on the judgment and experi-ence of experts. Any roll-up of program risks must be carefully done to prevent key risk issues from “slipping through the cracks.” Identify-ing program risk is essential because it forces the PMO to consider relationships among all risks and may identify potential areas of concern that would have otherwise been overlooked.

One of the greatest strengths of a formal, con-tinuous risk management process is the proac-tive quest to identify risk events for handling and the reduction of uncertainty that results from han-dling actions.

A program office has continuous demands on its time and resources. It is, at best, difficult, and probably impossible, to assess every potential area and process. To manage risk, the PMOs should focus on the critical areas that could affect the outcome of their programs.

Work Breakdown Structure (WBS) product and process elements and industrial engineering and manufacturing processes contain most of the significant risk events. Risk events are de-termined by examining each WBS element and process in terms of sources or areas of risk.

Broadly speaking, these sources generally can be grouped as cost, schedule, and performance, with the latter including technical risk.

Following are some typical risk areas:

• Threat. The sensitivity of the program to uncertainty in…