Attach 17 AOC Risk Mgmt Policy 2021.05.19.pdf

Policies and Procedures for Managing Risk in Construction Projects

May 2021

AOC Policies and Procedures for Managing

Risk in Construction Projects

Prepared for

Architect of the Capitol

Office of the Chief Engineer

Prepared by

McDonough Bolyard Peck, Inc.

Procon Consulting

May 2021

MBP/Procon | 3

Contents

1 Introduction

1.1 Construction Project Risk Management at AOC

1.1.1 Introduction

1.1.2 Purpose

1.1.3 Integration with AOC Project Management Manual and Other Manuals

1.2 Fundamentals of Project Risk Management

1.2.1 Introduction

1.2.2 Traditional Construction Project Risk Management

1.2.3 Project Risk Management

1.3 Applicable Policies, Standards and Best Practices

1.3.1 Introduction

1.3.2 GAO Schedule Assessment Guide

1.3.3 GAO Cost Estimating and Assessment Guide

1.3.4 Other Industry References

2 Project Risk Management Tools

2.1 Qualitative Risk Management

2.1.1 Overview

2.1.2 Qualitative Project Risk Management (Monitoring and Control of Risks)

2.1.3 Use of e-Builder Risk Management Module for Monitoring and Control of Risks

2.2 Quantitative Risk Assessment

2.2.1 Cost Risk Analysis

2.2.2 Schedule Risk Analysis

2.2.3 Integrated Cost-Schedule Risk Analysis

2.3 Project Definition Rating Index

2.4 Other Risk Management Tools

3 AOC Criteria for Selection of Project Risk Management Tools

3.1 Introduction

3.2 AOC Project Classification by Project Complexity (AOC Draft Memo dated October 13, 2020)

3.3 AOC Criteria for Selection of PRM Tools Based on Complexity, Cost, and Time Criticality

3.4 AOC Policy for Qualitative Risk Management

3.4.1 Purpose

3.4.2 AOC Criteria for Qualitative Risk Management

3.5 AOC Policy for Quantitative Risk Assessment

3.5.1 Purpose

3.5.2 How Results from Quantitative Risk Assessment Are Used in Budgeting Process

MBP/Procon | 4

3.5.3 Trigger Threshold for Quantitative Risk Assessment by Project Complexity, Cost, and Time Criticality

3.5.4 Exceptions to the Trigger Threshold for Quantitative Risk Assessment and Waiver Requirements ... 29

3.5.5 Timeline for Quantitative Risk Assessment

3.5.6 Underlying Information for the Development of Cost-Loaded CPM Summary Schedule

3.5.7 Confidence Level Selection

3.5.8 Use of Quantitative Risk Assessment Results

3.5.9 AOC Policy for Check Estimate

3.6 AOC Policy for Project Definition Rating Index

3.6.1 Purpose

3.6.2 Trigger Threshold for Project Definition Rating Index

3.6.3 Exceptions to the Trigger Threshold for Project Definition Rating Index and Waiver Process

3.6.4 Timeline for Project Definition Rating Index

4 Conclusion

5 Definitions and Terminologies

6 Acronyms

7 List of References and AOC Supplemental Guidebooks and Manuals

8 Industry References

MBP/Procon | 5

1 Introduction

1.1 Construction Project Risk Management at AOC

1.1.1 Introduction

At any given time, the Architect of the Capitol (AOC), manages hundreds of studies, design, and construction projects on behalf of the Congress, the Supreme Court, and certain other organizations, such as Union Station, that rely on the Capitol Power Plant for steam and/or chilled water. These projects range in size from a few thousand dollars to over $800 million and run the gamut in complexity ranging from routine repair, major renovation or alteration involving abatement and preservation, new construction, and modernization of an industrial steam and chilled water production and distribution system. AOC’s clients expect good designs, accurate cost estimates and delivery dates, quality work done safely and in compliance with health, environmental, preservation, occupant-imposed, and security requirements. Many projects are highly visible and subject to scrutiny by AOC’s Appropriations or Oversight committees, coverage by the news media, and audits by AOC’s Office of the Inspector

General (OIG) or the Government Accountability Office (GAO).

AOC’s projects are done in an environment involving risk and uncertainties that if not planned for and well-managed, can lead to failure to meet commitments; safety, health, environmental, or preservation mishaps; interruptions of client mission achievement, bad press, and loss of AOC’s credibility and trust.

Thus, it is important that from the very beginning, AOC identify and address the vulnerabilities of its projects to events that, if they occur, could have negative effects on objective accomplishment.

Likewise, it is also important that AOC identify and address opportunities that could have positive effects on its projects.

Risk management is a best practice by which AOC considers the likely impact of uncertainty and risks on accomplishing project objectives and proactively addresses them throughout a project. (Uncertainty reflects variations between actual and estimated conditions not attributable to a specific risk; one example would be the difference between the cost estimate and final bid amount.) AOC’s risk management program incorporates the best practices of the industry (Construction Industry Institute

(CII), Project Management Institute (PMI), AACE International (AACEI), Construction Management

Association of America (CMAA), and AOC’s own institutional experience. Initially, AOC implemented qualitative assessments of risk for its larger projects. Following a 2014 GAO review, AOC added a quantitative dimension to its risk assessment process for larger projects, incorporated GAO practices for cost estimating and scheduling, introduced formal partnering, and expanded its use of qualitative assessments to many more of its projects. AOC has made great strides to integrate project risk management throughout the organization, making it an integral part of the project and program management process. It is now AOC’s intent to expand project risk management to other AOC construction projects in a phased manner.

MBP/Procon | 6

1.1.2 Purpose

This guidance document is intended to set forth policies and procedures for construction project risk management to be implemented throughout program and project life cycles. These policies and procedures apply to AOC construction projects of all sizes, scopes, complexity, and delivery methods, except for extra low complex projects, for which Project Managers (PM) are to determine the most appropriate approach for identifying and managing risks. These policies and procedures align with GAO best practices to include, but not limited to, GAO-20-195G, “Cost Estimating and Assessment Guide: Best

Practices for Developing and Managing Capital Program Costs” (March 2020), and GAO-16-89G, “Schedule Assessment Guide: Best Practices for Project Schedules” (December 2015). For specific guidance on how to implement these policies and procedures, refer to AOC’s Best Practices Guide for

AOC Project Executives/Managers and Best Practices Guide for AOC Risk Management Consultants.

1.1.3 Integration with AOC Project Management Manual and Other Manuals

The policies and procedures set forth herein align with those set forth in following AOC documents.

• Project Management Manual (Manual 28-09), December 2013 (currently being updated)

• AOC Draft Memo 21-3, Project Complexity Rating (PCR) Tool, dated October 13, 2020 (currently being updated) (This document to replace section 1.2.1 of the AOC Project Management

Manual)

• AOC Project Submission Guidelines FY 2022, May 12, 2020 (updated annually)

• AOC Design Standards, December 2018

• AE Design Manual, December 2013

• Design Build Guidance

1.2 Fundamentals of Project Risk Management

1.2.1 Introduction

Risk management is a broad term. Therefore, it is important to make a distinction between the traditional use of the term “traditional construction project risk management” and “project risk management,” the practice of which is the specific focus of this document.

1.2.2 Traditional Construction Project Risk Management

Traditional risk management is associated with such practices as: taking steps to ensure proper allocation of construction risks to respective parties via the architect/engineer (A/E) contract documents; having appropriate construction insurance in place; implementing safety management on the jobsite; formal or informal programs for partnering and claims avoidance; and provision for a claims resolution process. Toward that end, contract documents are carefully reviewed to ensure such provisions as an owner’s right to correct defective work, an owner’s right to terminate for default or for convenience, indemnification provisions, lien Waiver provisions, warranties and bond/surety

MBP/Procon | 7 considerations are incorporated. While complexities of the individual construction projects might require crafting specific contract clauses, the management of risk through contract language has been a longstanding risk management method.

Figure 1 below lists some examples of traditional risk management components:

Examples of Traditional Risk Management

1-Other Contract Language/Provisions

• Claims resolution process

• Partnering

• Lien waiver provisions

• Indemnification provisions

• Safety and accident prevention plan

• Progress payment clauses

• Closeout provisions

• Substantial completion definition

• Force majeure definition

• Change order definition

3-Insurance Coverage

• General liability

• Builder’s risk

• Umbrella/excess coverage

• Workers’ compensation

• Contractor’s pollution liability

• Professional liability

• Miscellaneous coverage

• Project-specific requirements

4-Warranties

2-Surety Bonds

• Bid bonds

• Performance bonds

• Labor and material payment bonds

• Supply bonds

Figure 1 – Examples of Traditional Risk Management

The elements of traditional risk management are fundamental to the overall risk management in a construction project and are designed to protect the interests of various parties who have stakes in the project. The goal for each party involved (owners, contractors, subcontractors, etc.) is to allocate risk to the other project participants which would result in the least overall cost to the project. By its very nature, all instruments of traditional risk management are executed at the inception of the project, whether that point of inception be planning, design, procurement or construction. For instance, an owner will generally require payment and performance bonds to be submitted after a contractor’s proposal has been accepted and before executing the construction contract.

1.2.3 Project Risk Management

Broadly speaking, Project Risk Management (PRM) starts where the scope of traditional risk management ends. This is not to imply that the provisions of traditional construction project risk management (contract language, insurance, payment and performance bonds, claims resolution process, job safety plan, etc.) are not required and will not be in effect during the construction.

Traditional construction project risk management covers the insurable risks while project risk management addresses both insurable and uninsurable risks and proactively finds ways to treat them. In other words, PRM deals with any known and unknown risk, threat, or opportunity to the project that would have an impact on the project schedule, cost, safety, client operations or overall program

MBP/Procon | 8 objectives.

One of the most widely quoted definitions for project risk comes from the PMI’s Guide to the Project

Management Body of Knowledge (PMBOK) (2008/2017): “[project] risk is an uncertain event or condition that, if it occurs, has an effect on at least one objective. Objectives can include scope, schedule, cost, and quality”. The PMBOK describes PRM as “the process of conducting risk management planning, identification, analysis, response planning, and monitoring and control on a project. The objectives of PRM are to increase the probability and impact of positive events; and decrease the probability and impact of negative events in the project.”

AACEI, an organization recognized for over 50 plus years for its contribution in the areas of cost engineering, project controls and risk management, defines risk management as “the process of identifying risk factors (risk assessment), analyzing and quantifying the properties of those factors (risk analysis), mitigating the impact of the factors on planned asset or project performance and developing a risk management plan (risk mitigation), and implementing the risk management plan (risk control).”

Project risks are numerous and take various forms. Figure 2 below shows a partial list of typical risk types:

Examples of Risks in Construction Projects

Defective plans and specifications Permit requirements

Incomplete design Coordination with utility companies

Inadequate site investigation Working relationships among parties

Insufficient work definition Roles and responsibility among parties

Inadequate submittal review process Safety

Work quality Security

Preservation Unforeseen conditions

Unrealistic Schedule Inflation

Underestimated costs Weather and seasonal implications

Inadequate cost control Availability of site access

Inadequate project management control Local jurisdiction requirements

Inadequate change order management process Public relations

Inadequate RFI management process Availability of funding

Availability of skilled manpower Availability of equipment, fuel

Availability of acceptable materials Reliability of baseline schedule

Trade stacking Reliability of monthly schedule updates

Subcontractors’ ability to perform Sudden change or disruption to schedule

Suppliers’ ability to deliver Pandemics

Inexperience of project team Closeout requirements

Figure 2 – Typical Risks in Construction Projects

Project Risk Management Model: While several variations to the risk management model exist in the industry, there are essentially five steps in implementing a risk management program for construction projects. Figure 3 on the next page shows the project risk management model AOC will use for managing risks on all construction projects except for those that are of extra low complexity for which

PMs are to identify the most appropriate approach for identifying and managing risks. The specific

MBP/Procon | 9 action and documentation will vary by project complexity, cost, and criticality of schedule. Where applicable, e-Builder will be the tool AOC will use to manage and communicate risks.

Figure 3 – Elements of Project Risk Management

Step 1: Risk/Opportunity Planning - The first step in risk management is to define the scope of the risk management program so that it meets the program’s overall objective. Will it include quantitative risk assessment (QRA) with the intent of quantifying cost and duration contingencies? Will it include a risk management plan to address issues on an ongoing basis throughout the life of the project? Scope definition will, in turn, determine the type of expertise required in the project team. Once the risk planning efforts are complete and documented, implementation of the plan (in the form of Steps 2-5) can begin.

Step 2: Risk/Opportunity Identification - The second step is to identify all the potential risk and opportunity areas in the project. This can be done through a brainstorming session, interviewing, a workshop1, a questionnaire to project personnel, reviewing project documents such as budgets, schedules and contract clauses, visiting the project site, etc. Each risk item identified is then assigned a unique identifier and assessed as to the likelihood of its occurrence and the severity level of potential impacts and consequences.

Step 3: Risk/Opportunity Analysis - This step is where scope definition plays an important role. After

1 Details related to the Project Risk Workshop will be addressed in the Best Practices Guide.

MBP/Procon | 10 identifying all the potential risk and opportunity items, the agreed upon scope is implemented. This may come in the form of a quantitative risk analysis (such as running a Monte Carlo simulation2) to assess the overall cost and schedule impact of the identified risks and opportunities. Regardless of whether a quantitative assessment is done for a project, it will include a qualitative approach where a risk register is generated at project inception and the risk and opportunity item(s) are addressed on an ongoing basis

(weekly, monthly, or quarterly) until all the items identified have been resolved.

Step 4: Risk/Opportunity Response Planning - Once all the potential risk and opportunity items have been identified and their impacts assessed, the risk response options might be to:

• Transfer the risk item by delegating to another party,

• Reduce the risk by creative problem solving at a lesser cost or impact,

• Accept the risk by incorporating it in the project scope,

• Avoid the risk by severing the ties to the project or,

• Exploit the opportunity by taking action to increase the likelihood that positive aspects will occur

Of course, there could be a host of other responses depending on the project and its risk types.

Step 5: Risk/Opportunity Monitoring & Control - A comprehensive risk management program will monitor the risk and opportunity items continually by addressing status on a weekly, monthly, or quarterly basis. In this process, risk and opportunity items are reviewed to assess the status of mitigation implementation and reassessed for their likelihood of occurrence and impact. New risk and opportunity items will be identified and existing ones will be retired once a resolution is reached.

Depending on the project’s complexity, time criticality, or estimated hard construction cost, either a quantitative or qualitative risk assessment must be conducted prior to the project’s funding request.

1.2.3.1 Qualitative Risk Management

Qualitative risk management (QRM) involves taking steps to identify foreseeable risk events (things that could go wrong or be an opportunity) in the project, analyzing the potential impact of each risk event on the project, prioritizing them based on the severity of impact, creating problem solving strategies to mitigate (recover from) their impacts, and keeping an eye on each until the risk item is either averted, mitigated, accepted, or resolved or the opportunity item exploited. As shown in Figure 3 above, the process of risk/opportunity monitoring and control ensures continuity in its application throughout the project from the project’s beginning through construction. The PM, in concert with the Planning and

Project Development Division (PPDD), Energy and Design Services Division (EDSD), and the Project

Management Division (PMD), work with the risk register from the project’s inception. Shortly before construction starts or at construction start, AOC’s Construction Manager (CM) and PM work together to

2 Monte Carlo simulation is defined under Definitions and Terminologies section

MBP/Procon | 11 monitor, update, and manage the risk register.

The key to risk management is that it is always forward looking, anticipating risk and opportunity events well in advance of their occurrence and taking steps to prevent risks from happening, or at least reducing their impact by having a pre-prepared strategy or plan and plan/arrange for taking advantage of opportunities. This contrasts with issue management, which is reacting to and managing problems that have already surfaced. Put another way, without a risk management program, potential issues that could otherwise have been either eliminated or mitigated at a lesser cost or impact through risk management will turn into issues that must be dealt with at full cost or greater impact.

1.2.3.2 Quantitative Risk Assessment

If required, a quantitative risk assessment is typically performed at the conclusion of the design phase

(100% Construction Document (CD)) for Design-Bid-Build and at the Bridging Documents submission for

Design-Build and before issuance of the Project Cost Memorandum used for the project’s funding request. QRA is used to evaluate the viability of the project cost and/or time objectives. QRA may also be performed at periodic intervals during the course of the construction (such as prior to the beginning of a major long-term phased project) to verify that the cost and/or time objectives are still achievable given agreed upon risk responses. After gathering all known risk items and prioritizing them, quantitative risk analysis is performed to evaluate:

1. What is the probability that the project will be within the budget? This analysis will help to arrive at an estimate for required contingency funds. If contingency funding is not an option or if there is limited budget available, this offers guidance on de-scoping, phasing, or other options to consider. It may also identify additional costs that will be incurred, such as for risk mitigation or schedule extension, that need to be included in the budget submission.

2. What is the probability that the schedule deadlines will be met? This analysis will help to set a realistic project deadline. If the risk analysis shows that there is a very low probability of meeting the current deadline, the information provides the project team an opportunity to consider options for time contingency planning.

Along with evaluating contingency plans for time and cost overruns, decision makers may opt to perform quantitative risk analysis to evaluate such scenarios as the viability of the project itself and program alternatives. The steps required (risk/opportunity planning, risk/opportunity identification and assessment) to gather risk data for quantitative risk analysis are generally the same as those for qualitative risk management. It is what is done with the risk data, how it is analyzed and the specific purpose it serves that makes quantitative risk analysis different from qualitative risk management.

Should the results of a risk assessment indicate the budget and/or schedule may be difficult to achieve, establishing and agreeing upon scope contingency may be a possible approach. This would entail identifying one or more scope features that could be suspended/deferred pending the progress of the work. In such a situation, the client must buy-in and the project must be able to meet its overall

MBP/Procon | 12 objectives if the scope contingency must be used to meet budget or schedule.

1.3 Applicable Policies, Standards and Best Practices

1.3.1 Introduction

It is the intent of the AOC to adopt the following standards and best practices:

• Qualitative Risk Management: QRM within the context of this document is defined as taking steps to plan, identify, analyze, develop a response strategy; and monitor and control the risks and opportunities in the project through the use of a Project Risk

Register (PRR) developed by the project team and stakeholders using qualitative terms (high, medium, low) to define probabilities and impacts without simulating the results to arrive at statistical confidence levels. Use of a qualitative PRR is at the core of AOC’s risk management program.

The applicable standards and best practices for implementation of QRM on an AOC project will be a composite of AACEI, PMI, CMAA, and CII, tailored to AOC construction projects and included in this document. Practices contained in these other documents are not required for AOC unless they are specified in this policy document.

• Quantitative Risk Assessment: When required or deemed necessary, a quantitative risk assessment (e.g. cost risk analysis, schedule risk analysis or integrated cost-schedule risk analysis (ICSRA)) using a simulation model, is to be used to supplement

QRM. If a quantitative analysis is completed, the risks and mitigations developed during the QRA form the basis for the PRR. The resulting PRR shall be updated and managed throughout the remainder of the design and construction of the project.

QRA within the context of this document is defined as taking steps to identify, analyze, and perform quantitative risk analysis to either 1) ascertain the adequacy of cost (or schedule) contingency, and/or 2) calculate the amount of cost (or schedule) contingency needed for a project, at a given confidence level. QRA either builds on a risk register already prepared under QRM or provides the basis for the initial risk register to be used by the project team if a risk register has not already been developed.

For the implementation of QRA in an AOC project, it is the intent of AOC to adopt the best practices published by GAO. All other best practices or recommended practices serve as informational materials for performing QRA. They are not AOC requirements.

1.3.2 GAO Schedule Assessment Guide

The “Schedule Assessment Guide, Best Practices for Project Schedules” (GAO-16-89G), published by

MBP/Procon | 13

GAO, includes ten best practices. This guide, in its entirety, shall serve as the guiding document for developing the cost-loaded Critical Path Method (CPM) summary schedule required for performing QRA.

1.3.3 GAO Cost Estimating and Assessment Guide

The “Cost Estimating and Assessment Guide, Best Practices for Developing and Managing Program

Costs” (GAO-20-195G), published by GAO, includes two chapters that are directly applicable to the QRA:

• Chapter 11: Conduct Sensitivity Analysis

• Chapter 12: Conduct Risk and Uncertainty Analysis

1.3.4 Other Industry References

In developing its risk management program, AOC has been guided by the expertise of industry including organizations such as GAO, AACEI, PMI, CII and CMAA. AOC’s project risk management policies and processes reflect industry best practices. Specific references from each of these organizations can be found in Section 7. This policy adopts the GAO Guide sections identified above as requirements;

however, the other documents cited are for reference only and are not required.

MBP/Procon | 14

2 Project Risk Management Tools

2.1 Qualitative Risk Management

2.1.1 Overview

The steps that are common to both the QRM and QRA are discussed in Section 1.2.3 of this document.

The section below generally describes the key elements that are distinct to qualitative risk management.

For further detail on implementing a qualitative risk management plan, refer to AOC’s Best Practices

Guide for AOC Project Executives/Managers and Best Practices Guide for AOC Risk Management

Consultants.

2.1.2 Qualitative Project Risk Management (Monitoring and Control of Risks)

QRM involves taking steps to identify all foreseeable risk events (things that could go wrong and opportunities) in the project, analyzing the potential impact of each risk event on the project and prioritizing them based on the severity of impact, creating problem solving or opportunistic strategies to mitigate (recover from) their impacts, and keeping an eye on each until the risk or opportunity item is either avoided, reduced, transferred, exploited (in the case of an opportunity) or resolved.

The primary tool for monitoring and control of risks is the PRR which is developed and maintained throughout the life of the project. There are two ways in which risks and opportunities may be developed during the life of a project:

1. Qualitative Risk Management: The Planning Lead identifies early risks and the PM or

CM develops the PRR, identifying risks and opportunities, and developing response strategies. The PM and CM are encouraged to engage the larger project team and stakeholders through workshops, meetings, or e-mail to provide input to the PRR.

(See Section 3.4.2 for further discussion of the PRR.)

2. Quantitative Risk Assessment: The QRA process involves performing individual risk interviews with the subject matter experts to identify risks and conducting a group risk mitigation workshop to develop the risk response strategies. If the project team prefers to identify risks through a workshop rather than individual interviews, the PM must approve this. One of the outcomes of the QRA process is a robust Quantitative

PRR with risk mitigation strategies for each risk identified. The Quantitative PRR developed during the QRA process is then imported into e-Builder to generate the

[Qualitative] PRR for monitoring and control of the risks in the project.

The PRR remains in effect throughout the project and evolves over the life of the project, as risks and opportunities are identified, addressed, and then closed. An example of a PRR report used in monitoring and control of risk is shown in Figure 4 on the following page.

MBP/Procon | 15

Figure 4 – Sample e-Builder Project Risk Register Report

2.1.3 Use of e-Builder Risk Management Module for Monitoring and Control of Risks

AOC has implemented a Risk Management process within e-Builder for the planning and implementation of QRM. The PRR will be monitored and updated within the e-Builder Risk Management

Module (eRMM) process. Further detail about developing and updating the PRR in e-Builder is contained in the Best Practice Guides for AOC Project Executives/Project Managers and Risk Consultants. If a QRA is completed on a project, the resulting PRR updates are loaded into e-Builder and will be updated using qualitative risk analysis methods.

2.2 Quantitative Risk Assessment

2.2.1 Cost Risk Analysis

Cost estimates used to develop the project budget are what they are, estimates. Actual costs are frequently not the same as estimates. That is because there is an inherent variability in the components

(quantity and unit price of an item) that go into preparing the cost estimates. For instance, the rock excavation estimated at 20,000 cubic yards could end up being 24,000 cubic yards in actual construction. Similarly, an estimated 25,000 linear feet of pipe, priced at $20/linear foot may end up

27,350 linear feet in actual quantity and $25/linear foot in actual cost. In addition to the uncertainty in the estimate, projects face a host of external risks such as the permit delays, unforeseen subsurface conditions, labor shortages, etc., that can further impact the cost of a project.

The traditional way to account for the uncertainty and risks in a project is often to allocate a standard

MBP/Procon | 16 contingency3 in the budget. The exact percentage set aside may be different for each project and unique to each agency. However, if, as industry studies have reported, 9 out of 10 projects suffer cost overruns, the traditional way of establishing cost contingency is likely inadequate. Realizing this inadequacy in the budgeting process, an increasing number of agencies, public and private, have integrated QRA in helping them to set cost contingencies.

Figure 5 – Cost Risk Analysis Histogram (Monte Carlo Simulation)

Prior to construction and prior to construction budget submission, cost risk analysis helps to answer the question: how much contingency do I need given the risks and uncertainty in this project? When the project is already underway, cost risk analysis helps to answer the question: is the remaining contingency adequate to account for the risks and uncertainty still facing the project? Cost risk analysis provides the project team an opportunity to mitigate risks; and prepares project managers to develop risk responses to the needs of the project early and proactively.

3 It is noted that AOC uses 10% to 20% of project cost for contingency, when a QRA is not done.

Cost Risk Analysis: Determine if Cost Contingency for Project is Adequate

$3,800,000 $4,000,000

Distribution (start of interval)

H it s

0% $3,887,535

5% $3,960,475

10% $3,974,194

15% $3,984,615

20% $3,992,648

25% $4,000,387

30% $4,006,137

35% $4,012,131

40% $4,017,936

45% $4,023,366

50% $4,029,050

55% $4,035,716

60% $4,041,633

65% $4,047,068

70% $4,053,810

75% $4,060,637

80% $4,068,684

85% $4,077,091

90% $4,090,065

95% $4,106,309

100% $4,184,378

C u m u la ti v e F re q u e n c y

Entire Plan : Cost

Cost Contingency

(P80) or 80% Confidence

Project Budget

MBP/Procon | 17

2.2.2 Schedule Risk Analysis

The fundamentals behind the need for schedule risk analysis are the same as those described for cost risk analysis. Risks and uncertainty can cause the schedule to suffer slippages, resulting in overall project delays. At the planning stage, schedule risk analysis helps to answer the question: how much duration is optimum for my project, given the risks and uncertainty? If project duration is set too short, without adequately considering the risks and uncertainty in the project, bids may come in at a premium. If the duration is set too long, the project may incur unnecessary extended overhead costs. When the project is already underway, schedule risk analysis can answer the question: will my project complete on time given the risks and uncertainty still facing the project? It allows the project team to develop a risk response strategy in time that is appropriate and specific to the project.

Figure 6 – Schedule Risk Analysis Histogram (Monte Carlo Simulation)

2.2.3 Integrated Cost-Schedule Risk Analysis

Cost risk analysis generally calculates a cost contingency but does not account for the cost impact associated with schedule overrun. On the other hand, schedule risk analysis calculates a time

05-Mar-2022 21-Sep-2022

Distribution (start of interval)

H it s

0% 13-Jan-2022

5% 29-Mar-2022

10% 18-Apr-2022

15% 02-May-2022

20% 14-May-2022

25% 24-May-2022

30% 02-Jun-2022

35% 10-Jun-2022

40% 17-Jun-2022

45% 25-Jun-2022

50% 01-Jul-2022

55% 08-Jul-2022

60% 16-Jul-2022

65% 25-Jul-2022

70% 01-Aug-2022

75% 09-Aug-2022

80% 20-Aug-2022

85% 31-Aug-2022

90% 16-Sep-2022

95% 12-Oct-2022

100% 22-Feb-2023

C u m u la ti v e F re q u e n c y

Schedule Risk Analysis M1000 - Project Substantial Completion Date

(P80) or 80% Confidence Level Date

(20-Aug-2022)

Potential Schedule Impact or Schedule

Contingency

Deterministic Project Substantial

Completion Date (12-Apr-2022)

Schedule Risk Analysis: Determine Potential Impact to Project Schedule Due to Uncertainty and Risks

MBP/Procon | 18 contingency but does not consider the cost overrun associated with schedule overrun due to impact from uncertainty and risks. There is a more comprehensive approach to QRA which involves the use of cost-loaded CPM schedules for risk analysis. The cost loading is derived from the reconciliation of the

A/E’s estimate of project costs and an independent Check Estimate of project costs prepared by a third-party estimator when required or determined to be needed by the Director, PMD. The methodology, known as ICSRA, not only assesses the impact of risks and uncertainty to the schedule but also the cost impact to the project as a result of schedule delay.

Figure 7 – Benefits of Integrated Cost Schedule Risk Analysis

The use of ICSRA methodology to perform QRA has been accepted by GAO as having satisfied both the

Sensitivity Analysis and Uncertainty and Risks Analysis. The details as to how ICSRA is performed are discussed in the AOC’s Best Practices Guide for AOC Project Executives/Managers and Best Practices

Guide for AOC Risk Management Consultants. Typically, cost-loaded summary schedules are to be used for ICSRA. The summary schedules must generally conform to GAO’s Best Practices for Project

Schedules.

The level of effort needed to perform a schedule risk analysis and/or cost risk analysis separately for a project is nearly the same as that for integrated cost-schedule risk analysis. Therefore, given the benefit of ICSRA, AOC has adopted a policy to perform ICSRA for any project that is selected for QRA in lieu of separate cost risk analysis or schedule risk analysis. Exceptions may apply. Within the context of this document, except when noted, a decision to perform QRA for a project will imply a decision to perform

ICSRA. Should a PM believe that an ICSRA is not necessary when a QRA is required, a Waiver Request, along with the reasons, must be submitted to the Director, PMD, who can approve or deny the Waiver.

Schedule Time Contingency

(P80) or 80% Confidence

Deterministic Date

Why perform an Integrated Cost-Schedule Risk Analysis?

Cost Contingency

Deterministic Cost

(P80) or 80% Confidence

– Cost risk analysis calculates a cost contingency but does not account for the cost impact associated with schedule overrun

– Schedule risk analysis calculates a time contingency but does not consider the cost overrun associated with schedule slippage

MBP/Procon | 19

It is noted that, while the risks identified through the QRA process are analyzed for their impacts to the cost and schedule quantitatively, it should not be construed as a process that identifies only cost and schedule risks. The QRA process is intended to identify all foreseeable risks in the project such as safety, security, client, noise, client disruption, public relations, work quality, preservation, coordination, etc. All the risks identified in the project will be included in the PRR for monitoring and control of risks during construction.

2.3 Project Definition Rating Index

Project Definition Rating Index (PDRI) is a benchmarking tool developed by CII that uses a weighted checklist to help determine areas of poor definition that can create risk during project execution. The intent of the PDRI is to identify at the early stages of the project life cycle those factors which could negatively influence project outcomes and address them. For additional details on how to implement

PDRI, see the Project Management Manual.

There is an important link between PDRI, which happens early in the project and PRM. The outcomes of the PDRI in particular, the resulting Gap List and Action Item List, can be useful data to incorporate into the PRR. The Planning Lead and PM should ensure that risk data generated from the PDRI are incorporated into the PRR, as appropriate.

2.4 Other Risk Management Tools

AOC has identified additional best practices that project managers should consider when the size and complexity of a project demands additional tools beyond AOC’s standard project management practices.

These include constructability reviews, life cycle costing, and partnering. Not only do these tools have their own intrinsic value, but they also represent risk management tools that can be linked to and enhance overall PRM efforts.

Constructability reviews represent the application of construction knowledge, planning, and detailed design experience, scheduling expertise as well as a background in procurement strategies and field operations to enhance the quality of design documents. Constructability reviews reduce the risk of unanticipated changes during construction with their associated schedule and cost impacts. Although

OCE has an internal constructability review program, AOC requires these reviews by qualified independent third-parties for certain projects and that the results of the third-party review be considered along with the results of any internal reviews that were done. Third-party constructability reviews are required for design-bid-build projects that are high or extra high in complexity, are time critical, or estimated to cost more than $10 million in hard construction costs regardless of complexity.

For design-bid-build projects, PMs are to arrange for third-party constructability reviews at 100% CD.

Also, PMs are to see that the results of these reviews are considered along with any OCE internal constructability review results. PMs who would like to arrange for a third-party constructability review when it is not required must include this in the Project Risk Management Plan (PRMP) and ensure funding is available if needed. Constructability reviews are not done for design-build projects.

MBP/Procon | 20

If a PM believes a required third-party constructability review does not need to be done, a Waiver

Request must be submitted to the Director, PMD and the appropriate Jurisdiction representative along with the reasons.

Life cycle costing entails assessing life cycle costs/savings in making project trade-off decisions on various project parameters. During the programming and design stages, the project team considers alternatives for equipment and materials, and the planning/ design team performs life cycle cost analysis to determine the best value from a total cost perspective. Normally, life cycle cost analysis does not involve sophisticated probabilistic risk or sensitivity analyses. Probabilistic risk analysis captures the quantitative effects of various risks on total project value given the distribution of possible outcomes at various confidence levels. A deterministic sensitivity analysis identifies and quantifies the cost drivers that pose the most risk of impacts to the total project value when considered alone.

For projects that receive energy/sustainability scores of 70 and higher on the AOC Project Importance

Energy and Sustainability Scorecard and use energy savings to qualify for those ratings, the NIST 135 process for estimating cost savings related to energy savings is to be used. Probabilistic estimates of cost savings would only be required for projects that rely on non-appropriated funding, like Energy Savings

Performance Contracts, as called for by GAO’s Cost Estimating and Assessment Guide. Projects can always elect to do the probabilistic analysis, but that would be up to the project team. If so, a consultant would likely have to be retained to perform these analyses.

PMs will need to consider this issue early in the planning/programming stages so that funding can be requested if it is anticipated that sophisticated quantitative analyses like these are necessary in addition to a typical ICSRA addressed in this policy. If the PM believes that a probabilistic analysis is not necessary when non-appropriated funds are to be used, a Waiver Request must be submitted to the Office of the

Chief Engineer Deputy Director with the reasons.

Partnering is a structured process intended to align a project team (AOC and its contractors) around a shared objective of successful project delivery. Partnering promotes improved communication, accountability, strategy, and issue resolution, which results in a high performing team that can deliver project success. The partnership developed draws on the strengths of each organization to achieve a quality project done right the first time, within budget, on schedule, with an emphasis on safety, and with contractors making a fair profit. AOC teams can use partnering to help identify and manage risks.

PMs are to identify when and to what extent partnering is to be used when the PRMP is updated.

MBP/Procon | 21

3 AOC Criteria for Selection of Project Risk Management Tools

3.1 Introduction

Risk management is a key element of all AOC-managed projects regardless of dollar value or duration or the entity within AOC who is managing the project. PMs, their equivalents, and CMs are responsible for ensuring that uncertainties are considered and risks are identified, assessed, and managed for all projects throughout their entire life cycle, from early planning through construction and delivery, and that risk management is properly communicated and documented.

Risks and uncertainty are to be addressed before the start of any major project stage (Programming, Study, Design, or Construction). The nature, scope, and documentation of risk management will vary among projects based on such factors as their dollar value, time criticality, complexity, sensitivity, client impacts, possibility of encountering hazards or preservation mishaps, and vulnerability to security concerns. Although certain aspects of this policy do not apply to projects that are extra low in complexity, PMs are to determine the most appropriate approach for identifying and managing risks for these types of projects throughout their life cycles unless their estimated hard construction costs exceed

$10 million. While this situation may be rare, should a project extra low in complexity exceed this threshold, a QRA is required as well as other requirements for a project rated high in complexity.

3.2 AOC Project Classification by Project Complexity (AOC Draft Memo dated October 13, 2020)

The classification of an AOC project or estimated hard construction cost will determine the PRM tool to be implemented, be it PDRI, QRM or QRA. The AOC has developed a project rating system, the Project

Complexity Rating Tool, which classifies projects by their relative complexity: extra low, low, moderate, high and extra high.

The PCR Tool considers a number of project factors to arrive at a complexity rating: (1) total program budget; (2) scope definition and complexity; (3) acquisition strategies; (4) stakeholder involvement; (5) risk; and (6) security category (see Section 6). Based on the computed PCR, time criticality, or the project’s total anticipated hard construction cost, specific PRM tools are required. In addition, the timing for the implementation of the PRM tools is designated by project stage: Program, Study, Design, and

Construction.

MBP/Procon | 22

3.3 AOC Criteria for Selection of PRM Tools Based on Complexity, Cost, and Time Criticality

Table 1 summarizes the selection process for PRM tool:

Table 1 – Project Risk Management Tool Selection Process

PRM Tool

Criteria Description PDRI QRM QRA

Project

Complexity

Extra Low

Not required unless cost exceeds $10M

Per PM unless cost exceeds $10M

Required if cost exceeds $10M

Low

Not required unless cost exceeds $10M

Required - Informal QRM Required if cost exceeds $10M

Moderate

Not required unless cost exceeds $10M

Required - Informal QRM Required if cost exceeds $10M

High One required Required - Formal QRM Required if cost exceeds $10M

Extra High One required Required - Formal QRM Required if cost exceeds $10M

Time

Criticality

Mandated

Completion One required

Required - Formal QRM if cost is $10M or less

Required if cost exceeds $10M

*Note: If one of the above criteria is met, the PRM tool is to be used. Dollar values reflect the anticipated total hard construction cost (excludes soft costs and contingency) at the time of the funding request. QRA is required for any project for which the estimated hard construction cost exceeds $10 million. QRA is not required for a time critical project whose estimated hard construction cost is $10 million or less.

The matrix above shows that, while the selection of a particular PRM tool is based on the project’s complexity, time constraints, or estimated hard construction cost, it is AOC’s policy to implement QRM in some form for all projects to monitor and control risks throughout the project. For extra low complex projects, PMs are to select the most appropriate approach to identify and manage risks unless the estimated hard construction cost is estimated to exceed $10 million. In such a case, as well as for projects rated low and moderate in complexity, PDRI and QRA are required unless a waiver is approved.

3.4 AOC Policy for Qualitative Risk Management

3.4.1 Purpose

The purpose of QRM is to have a formal structure in place to identify every foreseeable risk or opportunity event in the project, analyze the potential impact of each risk or opportunity event on the

MBP/Procon | 23 project, prioritize risks based on the severity of impact, create problem solving strategies to accept, avoid, reduce, or transfer impacts or exploit opportunities, and keep an eye on each until the risk or opportunity has been resolved in some form. The key to risk management is that it is always forward looking, anticipating risk and opportunity events well in advance of their occurrence and taking steps to prevent them from happening, reducing their impact by having a pre-prepared strategy or plan, or taking steps to increase the likelihood of their occurrence (as in the case of an opportunity).

3.4.2 AOC Criteria for Qualitative Risk Management

Table 2 provides an overview of QRM tools and deliverables to be implemented based on project complexity. Essentially, QRM applies to all projects, but in addition to QRM, a quantitative risk assessment is required for all projects whose estimated hard construction cost exceeds $10 million unless a waiver is approved.

Table 2 – Qualitative Risk Management Tools and Deliverables

QRM Phase

Criteria Description

Risk/Opportunity

Planning*

Risk/Opportunity

Identification & Analysis

Risk/Opportunity

Monitoring/Control Monitoring

Tool

Project

Complexity

Extra Low PRMP Template Per PM unless cost exceeds $10M

Per PM unless cost exceeds

$10M

Per PM unless cost exceeds

$10M

Low PRMP Template Meetings, E-Mails or

Informal Risk Workshop

Weekly progress meetings/monthly/quarterly risk meetings

PRR in Excel or e-Builder

Moderate PRMP Template Meetings, E-Mails or

Informal Risk Workshop

Weekly progress meetings/monthly/quarterly risk meetings

PRR in Excel or e-Builder

High PRMP Template Formal Risk Workshop

Weekly progress meetings/monthly/quarterly risk meetings

PRR in e-Builder

Extra High PRMP Template Formal Risk Workshop

Weekly progress meetings, monthly/quarterly risk meetings

PRR in e-Builder

Time

Criticality

Mandated

Completion PRMP Template

Formal Risk Workshop if program cost is $10M or less

Weekly progress meetings, monthly/quarterly risk meetings

PRR in e-Builder

* The PRMP Template provides varying levels of detail to match the level of project risk management requirements based on the project complexity.

The PM is to indicate the approach for risk/opportunity monitoring and control in the PRMP. During planning and design, the PRR should be reviewed and updated as set forth in the PRMP for all projects except for projects extra low in complexity and expected to cost less than $10 million in hard construction costs. For projects that are extra low in complexity and estimated at under $10 million in

MBP/Procon | 24 hard construction costs, the PM must only complete a few items on the PRMP Template.

For all projects, except those that are extra low and under $10 million or less in estimated hard construction cost, during construction, weekly progress meetings should include a discussion of any significant risks associated with upcoming work per the contractor’s look-ahead schedules and any risks identified by the PM or CM that need to be discussed. For construction, the PRR is to be reviewed and updated as described below.

Extra Low Complexity Projects

For those projects identified as extra low complexity, QRM efforts will be identified by the PM in the

PRMP except if the estimated hard construction cost exceeds $10 million:

Table 3 – Extra Low Complexity Project Qualitative Risk Management Efforts

QRM Tool Description

PRMP PRMP Template. See AOC’s Best Practice Guide for Managing Risks in Construction

Projects for use by AOC Project Executives/Managers