Attachment 10 Implementation Risk Assessment Guidance.pdf

Navy ManTech Program

REV J OCT24

Implementation Risk Assessment Process Guidance Attachments:

1. Attachment 1: Implementation Risk Assessment – Project Category Definitions

2. Attachment 2: Implementation Risk Assessment – Risk Factor Definitions

3. Attachment 3: Implementation Risk Assessment – Project Category Risk Factor Weighting Factors

4. Attachment 4: Project Briefing Risk Assessment Template

5. Attachment 5: Implementation Risk Assessment – Project Risk Mitigation Plan Template

A. PURPOSE

The Navy ManTech implementation risk assessment process focuses on implementation and risks to implementation on a per Navy ManTech project basis, thereby improving the implementation probability of the project. The purpose of the implementation risk assessment process is to determine those factors that may prevent implementation of a technically successful ManTech solution and to mitigate the factors early in the ManTech project execution phase. It is the responsibility of the integrated project team (IPT) to use this guidance to maintain an updated implementation risk assessment evaluation and to develop details that support the rating for each risk factor.

The current implementation risk assessment will be presented at each program review to ensure that ManTech is in agreement on implementation progress with the acquisition and industry stakeholders whose representatives attend the program reviews. At the program review, the representatives will provide verbal feedback on their concerns and actions that should be taken by the IPT to improve or correct identified issues with each project. These suggestions will be recorded and provided to the IPT for action. The status of the recommendations must be addressed and briefed at the next program review.

Note: Because Rapid Response and Repair Technology Rapid Response projects are short in duration compared to the review cycle of ManTech projects, they will not be subject to a risk assessment at program reviews where they are presented. However, the principal investigator may provide an assessment for information purposes, if relevant.

B. IMPLEMENTATION RISK ASSESSMENT – PROJECT CATEGORIES AND RISK FACTORS

Using historical data, ManTech has defined six project categories and 11 individual risk factors as shown in the matrices in Attachments 1 and 2. While project categories and risk factors have been in use since October 2011, their definitions have been amended as necessary to optimize the risk assessment process.

C. INDIVIDUAL PROJECT IMPLEMENTATION RISK ASSESSMENTS

Each project will be subjected to an implementation risk assessment at appropriate program review meetings. When an IPT reviews the project, it should establish the project’s overall risk assessment score by using the following steps:

Determine the most appropriate project category (Categories 1 through 6) using the definitions in Attachment 1.

Use Attachment 2 to select the risk factor rating (RED, YELLOW, or GREEN) and assign a rating number (1 through 10) for each of the project’s risk factor categories.

o When a risk factor category is not applicable to the project, the risk factor cell should be “grayed” and the maximum score “10” should be assigned to that risk factor.

Use Attachment 3 to select the risk category weighting values for the project category selected.

Multiply each risk factor rating by the risk factor weighting values and sum to get an overall risk factor score.

REV J OCT24

Assign an overall project implementation risk assessment color using the following guidance:

o Scores greater than 700 will be assigned an overall GREEN rating.

o Scores between 400 and 700 will be assigned a YELLOW rating.

o Scores under 400 will be assigned a RED rating.

Implementation risk assessments are required for each project briefed at program reviews, if appliable (see Project Types - Data Call Requirements table). As specified in the Project Briefing Guidance, the briefer will use the Excel template shown in Attachment 4 to present the implementation risk assessment for the project.

At any point during the execution of a project, the IPT may determine that immediate action is required to mitigate technical, programmatic, or financial deficiencies that are threatening the viability of a project. The mitigating action should be documented in a Project Risk Mitigation Plan (Attachment 5) and forwarded to the appropriate Navy ManTech Program Officer and the Navy ManTech Support Team.

During the program reviews, the project’s current risk assessment will be briefed, and Navy ManTech, the Program Office, and the implementing industrial facility representatives will discuss the project implementation risk. Specifically, they will debate the appropriateness of the risk selection and ratings. If the participants believe the rating is incorrect, they can direct a change to the presented risk factor rating, and can request actions be taken prior to the next project review to mitigate issues. The comments and actions will be included in the minutes provided after the program review meeting.

It is anticipated that the COE and implementing industrial facilities will make progress on the risk assessment so that the overall implementation risk assessment rating improves from ‘YELLOW’ to ‘GREEN’ and, for projects with an initial ‘RED’ rating, from ‘RED’ to ‘YELLOW’ to ‘GREEN.’

Rev I OCT23

H

Note: The weighting factors for each risk factor for a specific project category provide a relative measure of importance of the risk factor toward implementation of the project. When a specific risk factor has no impact on the implementation of the project, the risk factor should receive the maximum rating (10). The overall risk assessment is based on this assumption.

Attachment 1: Implementation Risk Assessment – Project Category Definitions

1. Process Change – Under Cognizance of Industry (Only Company-Internal Resources Required)

• A production process change that does not impact the fit, form, or function of a platform component and does not require formal Navy approval. Production processing change, a material change in the production process, and above-the-shop floor production engineering / planning improvement are all examples that fall under this project category. The process should require only technical equivalency, technical maturity, and a strong business case to meet implementation requirements. There might be some equipment or implementation funding necessary (less than $100K). Only company-internal resources are typically required for implementation.

2. Process Change – Program Office /Technical Warrant Holder (TWH) Approval Critical

• A change that impacts a Navy-approved manufacturing process used in constructing naval components OR a material substitution for a previously approved baseline material. This category involves approvals by the TWH in accordance with existing specification requirements. An example is a project that advocates a new welding process. The approval of the new process requires formal written approval by the PO/TWH for implementation.

3. Process Change – Non-Navy ManTech Resource Critical (Equipment / Implementation)

• A production process change whose implementation is most critically affected by the availability of non-ManTech resources. The resources are likely either capital equipment or implementation funding which must be planned for and approved by senior management. Capital investment and/or implementation funding should be over $100K to be considered in this category.

4. Process Change – Multiple Critical Risk Factors

• A production process change that requires multiple critical risk factors to be considered. An example includes a project that requires a process certification to be successfully completed, but also includes significant capital and/or implementation funding to be secured in order for implementation to proceed. The critical risk factors that should be considered in this category are Technical Progress, Program Office Support, Technical Warrant Holder Approval, Major Milestones or Go/No-go Decisions, Capital Equipment / Facility Funding, and Business Case Strength / Benefit.

5. Design Change – Technical / Programmatic Focus

• A change to a platform component or structure that addresses an expressed technical deficiency and/or has a significant technical benefit driving its priority. The change will require Program Office and TWH approval or is a design change under the cognizance of an original equipment manufacturer (OEM). This category can result in an operationally improved product with no affordability benefit.

Extensive validation or certification testing may be an issue, as is a lack of platform Program Office and TWH support.

6. Design Change – Manufacturing / Affordability Focus

• A change to a platform component or structure that addresses design or manufacturability issues resulting in an unacceptable cost penalty. Projects in this category may be acquisition focused and have a strong return on investment or may not have a high acquisition Program Office priority, but have a significant life-cycle benefit. Extensive validation or certification testing may be an issue, as is a lack of platform Program Office and TWH support.

Attachment 2: Implementation Risk Assessment – Risk Factor Definitions

RED (1 to 3) YELLOW (4 to 6) GREEN (7 to 10)

TRANSITION PLANNING

Having a plan to achieve implementation is necessary to guide the project through its execution period. An approved TTP is an initial key to success. All parties (COEs, OEMs, and platform Program Office) should agree to the requirements to achieve implementation as well as to all of the resources and timing of those resources to achieve success.

A TTP has not been signed due to unresolved programmatic, technical, or financial issues.

or The TTP contains inaccurate project information for more than one program review.

or A project implementation plan does not exist or misrepresents current plans.

A current TTP is being routed for signature by all parties or a TTP is being revised.

or An implementation plan does not completely incorporate all necessary action by a project's participants.

A current TTP is signed by all parties.

and An implementation plan exists and accurately documents plans by all participants.

TECHNICAL PROGRESS

This factor measures the risk of achieving the stated technical objectives of the project. It measures the ability to maintain successful technical progress commensurate with the project plan and period of performance.

Achievement of critical technical goals has been delayed or has not been accomplished. Recovery is not certain.

or Technical progress is impacting the viability of the project.

Achievement of technical progress is lagging behind the project's plan. Recovery is achieveable and a recovery plan is in place to resolve the deficiency.

All scheduled technical goals have been achieved to date. Successful achievement of near-term technical requirements is anticipated.

PROGRAM OFFICE (PO)

SUPPORT

PO support is critical to the success of most Navy ManTech projects. This risk factor measures the programmatic and financial support of the PO for the project. For projects under the Process Change Under the Cognizance of Industry category, platform PO support may not be critical for implementation.

The PO withholds approval because of some programmatic or financial aspect of the solution.

or PO inputs are late, resulting in the delay of project activities.

The PO acknowledges the benefit of the project but does not commit to programmatic or financial support at this time.

or Required inputs from the PO do not support the project schedule.

Recovery is possible.

The platform PO fully supports the project and acknowledges future financial support, if required, for implementation of the solution.

and All inputs required by the PO are supplied in a timely manner.

Note: When the commitment and support of the PO have little or no impact on implementation of the project's solution, this risk factor should not be evaluated and should be marked "N/A" and left with a GRAY shaded cell.

Risk Factor Ratings Categories

Factor Definition and Categories Implementation Risk Factors Definition

Implementation Risk Assessment - Risk Factor Definitions

Attachment 2: Implementation Risk Assessment – Risk Factor Definitions Continued

TECHNICAL WARRANT HOLDER

(TWH) APPROVAL

When implementing a change to a platform system, the TWH must determine that the change does no harm and is fit to be used for operational service. TWH concurrence should be documented in writing.

The TWH withholds approval because of some technical aspect of the solution.

or TWH input is late resulting in delay of project activities.

The TWH will not or has not provided written support of the project and its planned solution.

or The TWH has not provided complete guidance on requirements needed to be met to achieve implementation.

The TWH fully supports the project in writing and is active in supplying requirements to be met for implementation.

Note: When the support and technical guidance of the TWH has little or no impact on the implementation of the project's solution, this risk factor should not be evaluated and should be marked "N/A" and left with a GRAY shaded cell.

MAJOR MILESTONE OR GO / NO-

GO DECISIONS

ManTech projects often require major milestones be achieved or go / no-go decisions be made in order to implement a solution.

Significant implementation risk can occur when major milestones are missed, occur at the end of a ManTech project, or are completed after the end of the project. Additional risk occurs when the major milestone or go / no-go decision are the responsibility of an entity outside of the ManTech community.

A major milestone or go / no-go decision is delayed due to an unresolved issue. The ability to recover is uncertain.

All major milestones remain to be achieved or progress is lagging on major milestones and go / no-go decisions.

or A milestone was not achieved, but a recovery plan is in place to address the issue.

Major milestones represent a valid assessment point and go / no-go decisions are well planned to assess project progress.

and Major milestones have been successfully met, and the project is on target to meet future milestones

Note: If this risk factor has no or little applicability to the implementation risk, this risk factor should not be evaluated and should be marked "N/A" and left with a GRAY shaded cell.

PRODUCT / PROCESS

CERTIFICATION

When implementing a change to a platform system, the component or material will be analyzed to determine that the change can meet all platform requirements. This can result in materials testing and evaluation, component prototype fabrication and performance testing, platform trials, etc. In such cases, the time and resources may be extensive. Extensive product / process testing programs occurring after the period of performance (POP) of the Navy ManTech project are a concern in this category.

Testing produces unacceptable results.

or An approved certification / validation plan is not available to begin testing and evaluation.

All or portions of the certification / qualification testing program will be executed after the completion of the Navy ManTech project.

or Testing is producing or has produced marginal results.

When testing is being conducted, all results are satisfactory.

and An approved qualification / certification test plan exists and is available for guidance.

Note: When certification / qualification is not required, this risk factor should not be evaluated and should be marked "N/A" and left with a GRAY shaded cell.

CAPITAL EQUIPMENT FUNDING

It is inherently risky when implementation is dependent on an implementation site’s capital investment. The severity of this risk depends on the amount, timing, and business case status of the investment. Capital equipment funding that does not require out-year planning or senior executive approval adds only moderate risk to implementation.

For projects that require a large investment, capital funding requirements are not known or not established.

or For projects that require a large investment, no source has been identified for the needed capital funding.

Capital funding support is expected, but no formal action has been taken to secure it.

Capital funding is available or a capital funding request has been inserted into the PO's or OEM's future capital budget.

or For projects requiring capital funding, the capital funding need is modest.

Note: When captial funding is not required, this risk factor should not be evaluated and should be marked "N/A" and left with a GRAY shaded cell.

NON-NAVY MANTECH

IMPLEMENTATION FUNDING

This is funding required to implement the ManTech solution into production (funding not provided by ManTech). It does not include capital equipment, but includes nonrecurring engineering costs, certification or verification testing programs, prototype construction, training, and start-up production costs.

Important contributors include whether the funding is secured, planned, or unplanned. The certainty of the commitment to the amount and the identification of the sources are important as well.

Implementation funding requirements are not known or not established.

or No source has been identified for the needed implementation funding.

Implementation funding support is expected, but no formal action has been taken to secure it.

(Formal action is considered taken when documentation requesting funding exists.)

Non-ManTech implementation funding is available or has been inserted into the PO's or OEM's future capital budget.

or Implementation funding is modest (not requiring senior management approval or out-year planning).

Note: When Non-Navy ManTech funding is not required, this risk factor should not be evaluated and should be marked "N/A" and left with a GRAY shaded cell.

BUSINESS CASE STRENGTH /

BENEFIT

For affordability projects, a business case is the most critical factor. Initially, a strong return on investment (ROI) is essential and, as the project is executed, a strong business case is necessary. Risk in this category increases when the business case assumptions are proven incorrect and the financial benefit drops. An initial ROI greater than 1.0 is desireable. If the ROI is low, some additional benefit should be provided to consider continuation of the project.

For non-affordability-focused project, the business case is required, but is not a critical parameter in judging the viability of a project.

For affordability-focused projects, an ROI is based on unsubstantiated assumptions.

or The ROI (ROI<0.5) and/or business case is low and there are no other benefits that would justify OEM implementation.

For an affordability-focused project, the business case may not be sufficient to warrant the OEM to consider implementation.

or The business case is based on broad assumptions.

or A business case needs to be revised before it can be used to justify OEM implementation.

For an affordability-focused project, a valid and positive business case, based on sufficient details, exists.

Note: To determine the reasonableness of the affordability benefit, an ROI greater than 1.0 and an acquisition business case analysis that pays back the Navy ManTech investment within three years (five years for carriers) is considered a reasonable return. Life-cycle-focused projects that have a 10X return on the Navy ManTech investment over the life of the class is considered a reasonable return.

TECHNOLOGY / PRODUCT

MATURITY

A ManTech solution may result in a new process or product technology that must be implemented into production or aboard the platform. These solutions may involve immature manufacturing processes or technology that has not been utilized extensively in the manner planned for by the project. This category measures the robustness and reliability of the process or product as it is developed throughout the project.

The solution represents the first-time application of the technology in production. (BENCHTOP /

BREADBOARD DEMONSTRATED)

or Laboratory-scale demonstration has yet to be performed. Learning curve duration and production viability are in question.

The application of the technology has been demonstrated in a non-production environment and not at the implementing facility.

(LABORATORY VALIDATED)

or A modest learning curve is expected.

or Equipment maintenance plans require attention, but maintenance of the equipment is not expected to be a time-critical issue.

The technologyt has been matured in other industries. (PRODUCTION PROVEN) or Little or no learning curve is anticipated and maintenance aspects of the new technology are not likely an issue.

COMMERCIALIZATION PARTNER

PARTICIPATION

New manufacturing tools or new processes may require components that have not been made before in commercial industry. This risk factor addresses the plan to commercialize the product or process. Important considerations in this factor include whether a commercial source has been developed and is capable of meeting the demands that may be required of it once the solution is fully implemented. If cost savings are considered important, then the commercial source must provide documentation that it can produce the product or process at the cost being used in the business case analysis.

No participating commercial partner has a been identified.

or An identified or participating commercial partner has stopped active support of the project.

A participating commercial partner has not been secured.

or Active participation of a commercial partner has not occurred.

A commercial vendor has agreed to participate in the development of the equipment for the project. The vendor intends to supply the final project's hardware.

Note: If a commercial partner is not required, this risk factor should not be evaluated and should be marked "N/A" and left with a GRAY shaded cell.

Attachment 3: Implementation Risk Assessment – Project Category Risk Factor Weighting Factors

Navy ManTech Program - Implementation Risk Assessment

Project Categories, Associated Risk Factors, and Associated Weighting Factors

Risk Factors

1 2 3 4 5 6 7 8 9 10 11

Project Categories T ra n s it io n

P la n n in g

T e ch n ic al

P ro g re ss

P ro g ra m O ff ic e S u p p o rt ch n ic al

W a rr an t H o ld e r

A p p ro va l

M aj o r

M il es to n e o r

G o

/N o

-G

D ec is io n s ro d u ct

P ro c es s C e rt if ic at io

C ap it al

E q u ip m en t / F a ci li ty

F u n d

N n -N av y

M a n T e ch

T ra n s it an d

Im p le m en ti n g

F u n d

B u s in es s

C a s e

S tr en g th

B en ef it ch n o lo g y

P ro d u c t M a tu ri ty

C m m e rc ia liz at io n P a rt er ar ti c ip at

1 Process Change - Under Cognizance of Industry 18 17 3 7 6 6 11 11 10 5 6

2 Process Change - PO / TWH Approval Critical 14 14 11 13 7 7 6 9 9 5 5

3 Process Change - Non-Navy ManTech Resource Critical 12 12 8 8 7 5 13 13 10 6 6

4 Process Change - Multiple Critical Risk Factors 12 12 10 10 7 6 10 10 10 7 6

5 Design Change - Technical / Programmatic Focus 14 14 15 8 6 10 7 10 8 4 4

6 Design Change - Manufacturing / Affordability Focus 12 12 14 8 6 14 4 8 14 4 4

Attachment 4: Project Briefing Risk Assessment Template

Title:

PROJECT RISK ASSESSMENT TREND Project Category:

Status (Pending, Active, Completed): Pending Active Active Active Completed

Presentation Date: MM/DD/YYYY MM/DD/YYYY MM/DD/YYYY MM/DD/YYYY MM/DD/YYYY Risk Factors

Transition Planning

Technical Progress

Program Office Support

Technical Warrant Holder Approval

Milestone or Go/No-Go Decision

Product / Process Certification

Capital Equipment / Facility Funding

Non-Navy ManTech Transition and Implementation Funding

Business Case Strength / Benefit

Technology / Product Maturity

Commercialization Partner Participation

OVERALL ASSESSMENT OF RISK

REV I OCT23

Attachment 5: Implementation Risk Assessment – Project Risk Mitigation Plan Template

Implementation Risk Assessment Project Risk Mitigation Plan

Date: ________________________________________________________________________

Project Title: _________________________________________________________________

Project Number: ______________________________________________________________

COE: ________________________________________________________________________

Platform: _____________________________________________________________________

Implementing Organization: _____________________________________________________

Date of Program Review Briefing (Resulting in ‘RED’ Overall Risk Assessment Rating):

Risk Factor(s) Requiring Mitigation:

1. xxxx

2. xxxx

Deficiency Causing Risk Factor Assessment Rating:

1. xxxx

2. xxxx

Description of Risk Mitigation Approach (Assign Dates / Responsible Parties to Proposed Actions):

1. xxxx – Responsible Party (Date)

2. xxxx – Responsible Party (Date)

Project Principal Investigator: ______________________________________ Date: __________

ManTech Program Officer: ________________________________________ Date: __________

Projects Receiving a ‘RED’ Overall Risk Assessment Rating:

1. The IPT should determine if a Project Risk Mitigation Plan (Attachment 5) should be submitted to the respective ManTech Program Officer (with a copy to the Navy ManTech Support Team) to alert of specific actions to be taken to mitigate the identified technical, programmatic, or financial issue affecting implementation. The Risk Mitigation Plan should detail immediate actions to be taken to correct a recently incurred critical deficiency in the project.

2. During the next Risk Assessment at a program review, the COE should report on the mitigating actions that were taken.

3. Projects having a ‘RED’ overall Risk Assessment rating for more than one program review are subject to termination, at the discretion of the ManTech Program Officer.