Att_5_TTP_Impact_Analysis.pdf

TTP for Impact Analysis v1.0

March 2015

AIR FORCE TACTICS,

TECHNIQUES AND PROCEDURES

TTP FOR IMPACT ANALYSIS V1.0 MARCH 2015

VERSION 1.0 – PRE-PUBLICATION

UNCLASSIFIED ii

THIS PAGE INTENTIONALLY LEFT BLANK

UNCLASSIFIED iii

Support

For additional information or questions about this document refer to:

https://cs1.eis.af.mil/sites/AFExec_UST/SitePages/Home.aspx

Series

This series of Air Force (AF) Tactics, Techniques and Procedures (TTP) includes:

UNCLASSIFIED iv

Version Management

Name and Rank Date

Summary of Changes

Name of Change Author Approval Date

UNCLASSIFIED v

Table of Contents

Support ........................................................................................................................................... iii

Series .............................................................................................................................................. iii

Version Management ..................................................................................................................... iv

Summary of Changes ..................................................................................................................... iv

1. Introduction

2. Service Development and Delivery Process (SDDP) Applicability

3. Impact Analysis Steps

3.1 Entry Criteria ……………………………………………….…………………………...2

3.2 Step 1: Initiate the Impact Analysis

3.3 Step 2: Perform Status Quo Analysis

3.3.1 Step 2a. Requirements Analysis

3.3.2 Step 2b. Schedule Analysis

3.3.3 Step 2c. Cost Analysis

3.3.4 Step 2d. Risk Analysis

3.3.5 Step 2e. Benefits Analysis

3.3.6 Step 2f. Notify Senior Leadership

3.4 Step 3: Brainstorm Alternatives

3.5 Step 4: Evaluation of Alternatives

3.6 Step 5: Present the Impact of the Solutions

3.7 Exit Criteria

Appendix A – Sample Impact Analysis for a Notional Program

The Supply Maintenance System (SMS) Overview

The Triggering Event

Step 1

Step 2

Requirements Trace

Schedule Analysis

Cost Analysis

Risk Analysis

Benefits Analysis

Notify Senior Leadership

Step 3

UNCLASSIFIED vi

Brainstorm Alternatives

Filter Alternatives

SMS Alternative to Perform the EBS interface manually

Step 4 and Step 5

Appendix B – Acronym List

Appendix C – Glossary

Appendix D – References

List of Figures

Figure 1: Impact Analysis Relates to SDDP Documentation throughout Lifecycle

Figure 2: Status Quo Analysis Incorporates the Triggering Event

Figure 3: SMS Estimates for Status Quo Shows no Change in Base Year (BY) Dollars across Fiscal Years (FY)

Figure 4: Example Presentation of the Status Quo Analysis

Figure 5: Requirements Affected by the EBS Interface through Failed Interface in ITFR 5

Figure 6: Only Release 2 Build and Test are Impacted, but Not Delayed ………………..……..11

Figure 7: EBS Delays Capability Delivered by M-Solution

Figure 8: DOTMLPF-P Milestones are aligned for successful execution

Figure 9: Current Program Status Quo Execution Has Same Estimated Cost

Figure 10: Presentation of the Status Quo for the SMS Example

Figure 11: Potential Alternative Examples

Figure 12: Eliminate Alternatives

List of Tables

Figure 1: Impact Analysis Relates to SDDP Documentation throughout Lifecycle

UNCLASSIFIED 1

1. Introduction

This Impact Analysis Tactics, Techniques and Procedures (TTP) provides step-by-step instructions to perform a fact-based analysis of potential impact and proposed alternatives to an initiative. Impact Analysis is performed using existing configuration documentation developed during the initial business process analysis through the development and delivery process to evaluate the result of changing circumstances; as well as to consider alternative solutions to provide optimal value. Impact Analysis is conducted from a cost and schedule performance perspective to the project Materiel (M)-Solution, as well as to the full set of Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities, and Policy (DOTMLPF-P) activities. The Impact Analysis process results in a quantitative assessment of the value of the initiative alternatives as represented in the return on investment (ROI).

2. Service Development and Delivery Process (SDDP) Applicability

The Air Force (AF) Service Development and Delivery Process (SDDP) requires necessary and sufficient project requirements, documentation, and planning (Figure 1) to deliver user-centric solutions that support Warfighter needs.

Figure 1: Impact Analysis Relates to SDDP Documentation throughout Lifecycle

The Impact Analysis should include tradeoffs and effects against each of the DOTMLPF-P capabilities, conducted within the Integrated Project Team (IPT) or the Integrated Program Office (IPO) construct.

UNCLASSIFIED 2

3. Impact Analysis Steps

The five-step process, shown in Table 1 and detailed in the next sections, provides a clear methodology for conducting an Impact Analysis.

Impact Analysis Steps Description

Step 1: Initiate the Impact Analysis

Based on new information that a project may no longer be executable as planned, documents are collected and a task oriented team is formed to perform the Impact Analysis.

Step 2: Perform Status Quo Analysis

The Impact Analysis Team evaluates the effect of the new information on the performance, schedule, and cost of the project. Project related issues are identified within the context of performance, schedule, and cost.

Step 3: Brainstorm Alternatives

The Impact Analysis Team identifies many potential problem resolutions, and sorts these alternatives to identify the most promising approaches that require further analysis.

Step 4: Evaluation of Alternatives

The impact of each remaining (down-selected) alternative is evaluated against the performance, schedule, and cost of the project.

Step 5: Present the Impact of Solutions

The selected alternatives are presented to Senior Leadership with the impact of each alternative shown against the existing configuration managed project.

Table 1: Performing the Impact Analysis – Steps 1through 5

3.1 Entry Criteria

Impact Analysis requires detailed project information related to the event. General entry criteria include:

Triggering Event - An event that is perceived to impact performance, schedule and/or cost

Defined and approved needs, capabilities, or requirements across the DOTMLPF-P spectrum contained in the SDDP content listed in Figure 1

Approved baseline documents including the Performance Reference Model (PRM) and all derived/developed requirements documentation

Lifecycle documentation including the latest approved configuration managed Schedule and Cost documents

3.2 Step 1: Initiate the Impact Analysis

A Triggering Event has the potential to alter, positively or negatively, the planned execution of the project in relation to performance, schedule, and/or cost, and causes the IPT/IPO to initiate the Impact Analysis process. Common Triggering Events include resource changes, new direction or constraints, new/changed requirements, schedule changes, testing failures, development issues, test design, program management, external program impacts, and interface

UNCLASSIFIED 3

failures. The IPT/IPO briefs Senior Leadership with the information concerning the identified triggering event and the anticipated requirement for an Impact Analysis. If approved, Senior Leadership will charter an Impact Analysis Team and provide guidance concerning the necessary considerations and constraints. The Impact Analysis Team will use the latest configuration documentation including the latest and most detailed requirements specifications, schedule, and estimated costs. As the Impact Analysis is executed, the team may be augmented with additional documentation or expertise as required. Table 2 provides a reference of team skills applicable to general triggers.

Trigger Initial Team (Skills Applicable)

P ro gr am

M an ag er

S p on so r R ep re se n ta ti ve

U se r S

M E ys te m s E n gi n ee r

S of tw ar e E n gi n ee r

F in an ci al M an

C os t E st im at or ch ed u le r

R is k M an

T es t M an

R eq u ir em en ts A n al ys t

L og is ti ci an

M an p ow er

T ra in in g

In fo rm at io n

A ss u ra n ce

P la tf or m E n gi n ee r

E n te rp ri se

A rc h it ec t

Resource Change

New Direction or Constraints

Requirements Changes

Schedule Mod Testing Failures

Development Issues

Test Design Mod Program Management

External Program Impacts

Interface Failures

Table 2: The Impact Analysis Team Skills

3.3 Step 2: Perform Status Quo Analysis

Performing the Status Quo analysis determines the altered value to the Air Force based on the impact of the Triggering Event. The Status Quo consists of the existing approved baseline adjusted only for the factual effects of the Triggering Event as shown in Figure 2. It does not include any proposed changes or solutions that will be considered as responses to the Triggering Event. The team begins the Status Quo analysis by reviewing the affected applicable

UNCLASSIFIED 4

requirements, the schedules, the estimate and cost derived artifacts, risk, and calculating the altered ROI. As analysis proceeds, the team may have to iterate through the following sub-steps.

Figure 2: Status Quo Analysis Incorporates the Triggering Event

3.3.1 Step 2a: Requirements Analysis

Requirements Analysis identifies the point(s) of impact in the requirements documentation that are directly affected by the triggering event. Requirements Analysis applies at any stage of the initiative lifecycle and applies to current program baseline from the Problem Statement through the PRM to the Requirements Repository. If the Requirements Repository is developed, a requirements traceability matrix (RTM) should be generated to identify all related requirements for impact evaluation. The structure of a standard requirements repository and an example RTM are described in more detail in the TTP for Mission Applications Requirements Capture and Traceability. The resultant identified requirements are then evaluated to determine which specific requirements are actually affected by the triggering event in order to determine the collective impact. This process allows existing requirement validation and/or associated new requirement implementation. Upon completion, the requirements analysis captures the performance impact to the initiative capability based on the triggering event.

3.3.2 Step 2b: Schedule Analysis

Schedule analysis captures the tasks in the approved schedule that affect the execution plan. The TTP for Integrated Master Schedule describes the development of schedules, and the same expectations apply to schedules modified for the Status Quo. The team will assess the impact to the DOTMLPF-P Implementation Plan, the M-Implementation Plan and/or the materiel acquisition program IMS due to the triggering event. The relationship of the M-Solution milestones against the DOTMLPF-P Implementation Plan milestones should also be evaluated such that any dependencies are understood and documented. After all of the impacted tasks in the schedules have been identified, the schedules are updated based only on actual changes realized to date and those directly caused by the triggering event. While the duration of tasks may change, the Status Quo schedule does not change the intended approach or products, so the tasks in the schedule are not adjusted in scope. Upon completion, Status Quo IMSs are updated and linked from the materiel acquisition program to the M-Implementation Plan to the

UNCLASSIFIED 5

DOTMLPF-P Plan milestones. The schedules contain updated tasks, resources and durations used in the following steps.

3.3.3 Step 2c: Cost Analysis

Once the schedule analysis is complete, the resource loading of the Status Quo schedule(s) provide a cost estimate for the program (see TTP for Schedule-Based Estimating). This estimate is compared to the current configuration managed schedule-based estimate including both the DOTMLPF-P Implementation Plan and the M-Implementation Plan schedule based estimates. Upon completion, deviations between the Status Quo and the approved initiative are identified as shown in Figure 3. The second table illustrates a potential deficit for a similar initiative.

Figure 3: SMS Estimates for Status Quo Shows no Change in Base Year (BY) Dollars across Fiscal Years (FY)

3.3.4 Step 2d: Risk Analysis

The team updates the risk register (in accordance with the TTP for Integrating Risk into Program Plans) with any changes to risk, based on the Status Quo, including new risks and updated likelihood or consequence to existing risks. The updated risk register provides additional input to the Impact Analysis, but the Status Quo schedule does not include new risk mitigation activities. Upon completion, the team provides a summarized description of the changes in risk from the configuration managed risk register to Senior Leadership.

3.3.5 Step 2e: Benefits Analysis

Benefits should be re-evaluated based on the Status Quo Impact results. Capabilities delivered, and the timeline for achieving benefits are both factors in the calculation of benefits. Upon completion, time phased benefits are evaluated against updated costs to develop an ROI for the initiative.

3.3.6 Step 2f: Notify Senior Leadership

The IPT/IPO briefs the Status Quo documentation to Senior Leadership for guidance, as shown in Figure 4. The Status Quo presentation highlights to Senior Leadership the changes caused by the Triggering Event against the program baseline. This is an opportunity for senior leaders to provide the Impact Analysis Team with amplifying guidance, based on program objectives and tradeoffs, to further develop the Impact Analysis IAW this TTP or if the Status Quo shows small and acceptable changes in the program, may direct the Impact Analysis be terminated and the Status Quo option implemented.

UNCLASSIFIED 6

Figure 4: Example Presentation of the Status Quo Analysis

Documentation associated with the Status Quo analysis is retained for future use in the Impact Analysis. At this point in the impact analysis, the team has completely assessed the impact of the trigger event on the current baseline.

3.4 Step 3: Brainstorm Alternatives

Based on Senior Leadership guidance and an understanding of the full effect of the triggering event on the initiative the Impact Analysis Team brainstorms alternatives to the Status Quo.

Brainstorming should be conducted with identified possibilities captured before any alternative evaluation is performed. The following considerations should include but not limited to:

Modify planned DOT_LPF-P changes/tradeoffs to reduce requirements on the M- Solution

Consider design solutions alternatives that could save time or money Shift development to (more) Commercial Off-The-Shelf (COTS) or Government Off-

The-Shelf (GOTS) to save time and/or money Consider long and critical path tasks to re-sequence to save time and/or money Introduce parallelism or consolidation into planned events: Opportunities for development and testing consolidation and acceleration Adjust or delay the phasing of development/deployment to accommodate funding profile Identify high cost and low value requirements for elimination

Once the brainstorming is complete the alternatives are filtered to remove inappropriate, infeasible, or clearly suboptimal alternatives. At the conclusion of this step the team produces a

MS B IOCMS C FOC

Status Quo - Current Program Status after EBS Delay

Cost/Benefit Analysis to the Air Force: ROI 3.107:1

DOTMLPF‐P Effects:

• Use manual Workarounds.

• Interface finally enabled by the EBS Upgrade program.

Requirements Capability:

• SMS functionality is complete

• EBS interface capability is not functional until EBS upgrades are complete

• User must provide work around for EBS interface

Cost 121k 785k 1,382k 837k 411k 386k 125k 85k 1,200k

2013 2014 2015 2016 2017 2018 2019 2020 2021 - 50 FY

Risk:

1. Added risk of additional EBS upgrade delays.

2. Test SMS / EBS interface by simulation.

No Change to Current Initiative Schedule for either the Materiel Program or the DOTMLPF‐P Plan

Program and Initiative Cost is unchanged

Costs Benefits ROI

Baseline $144,446,283 $449,227,941 3.110

Status Quo $144,446,283 $448,829,791 3.107

UNCLASSIFIED 7

list of alternatives for further analysis. Furthermore, a list of rejected alternatives should be identified with specific elimination explanations.

3.5 Step 4: Evaluation of Alternatives

The alternatives that are not eliminated by initial screening are further analyzed. For each alternative the project related changes to program artifacts should be identified and applied as follows:

1. Requirements are changed or reallocated based on the alternative.

2. Changes to the WBS (See TTP for Work Breakdown Structure) and design are traced to the COA Components.

3. Tasks in the IMS associated with the updated design, requirements, and WBS are identified and removed, added or changed to implement the alternative.

4. Any additional or reduced risk associated with the alternative is identified; risk mitigation strategies (if any) are developed and added to the IMS.

5. The Schedule milestones for the M-Solution are adjusted for the alternative and evaluated against the DOT_LPF-P milestones to ensure that the synchronization of the initiative is maintained.

Once the changes have been posted to the program planning documents, the analysis performed on the Status Quo, in Step 2, is iterated for each alternative (Steps 2a-2e). Each change to the baseline requirements documentation is recorded in the alternative copy of the documentation. If the RTM has been developed the requirements and linkages are revised to accurately represent the alternative. All associated schedules (DOTMLPF-P Implementation Plan, M- Implementation Plan, and Acquisition Program IMS) are updated with the tasks necessary to implement the alternative. The schedule-based estimate for the alternative is generated, and risks are reviewed. Where the Status Quo analysis merely identified changes in risks, the alternative analysis includes risk mitigation activities in the schedule and cost. The benefits of each alternative are captured and quantified for inclusion in the ROI. Upon completion, each alternative is described against the program baseline as shown for the Status Quo in Figure 4.

3.6 Step 5: Present the Impact of the Solutions

The Impact Analysis concludes when the IPT/IPO briefs the preceding analysis to Senior Leadership for decision. The decision package should include the Status Quo analysis; all alternatives considered and the reason for eliminating any from further analysis, and the completed analysis for the remaining alternatives.

The work performed by the Impact Analysis Team is retained in order to rapidly implement the decision of the Senior Leadership. Retained work will be valuable in performing revisions if directed, or in performing future analysis.

3.7 Exit Criteria

The completion of the Impact Analysis process provides details and COAs for Senior Leadership decision making:

Analysis of decision tradeoffs for the decision maker

▪ Comparison of alternative values to the AF against the prior baseline (ROIs)

▪ Review of the impact on Mission Capabilities for the AF

UNCLASSIFIED 8

▪ Risks, with mitigation activities for each alternative

▪ DOT_LPF-P

Detailed implementation instructions for execution of Leadership selected alternative for:

▪ Requirement and design changes

▪ Schedule adjustments for the program and initiative, including risk mitigation activities

▪ Costs associated with the event or alternatives

A cost, schedule, performance characterization of the Impact of an event or change on a program or initiative

Feasible alternatives documented, quantified, and based on factual data

The completed Impact Analysis provides all of the analysis necessary for Leadership to make an informed decision concerning the future of the initiative.

UNCLASSIFIED 9

Appendix A – Sample Impact Analysis for a Notional Program

The following example provides a notional application of Impact Analysis to a fictional SDDP initiative.

The Supply Maintenance System (SMS) Overview

The Supply Maintenance System (SMS) is an Information Technology (IT) Business System that plans and supports maintenance activities. It is currently in development (SDDP Step 4) with all of the appropriate approved documentation, and is therefore under the management of a program manager within the construct of an Integrated Program Office (IPO). SMS is being completed in four releases and deployed in seven sites. During SMS development a feeder system, the EBS upgrade program, has been delayed. As a result of the delay the anticipated automated interface with EBS will be delayed for 18 months. EBS is one of 12 systems that provide data in support of Release 2 of SMS. EBS data is required in approximately 10% of maintenance work orders supported by SMS.

The Triggering Event

Once the issue is discovered by the development team, the team follows steps 1-5 of this TTP to determine if the EBS interface unavailability has any effect on the SMS program.

Step 1

The development team determines that the EBS interface is necessary in order to meet the approved program requirements. Given that the interface cannot be produced within the existing approved schedule and budget, the issue is raised to the IPO. Based on Senior Leadership guidance the Program Manager (PM) leads an Impact Analysis team to evaluate the impact and examine alternatives.

Step 2

Requirements Trace

The requirement for SMS to interface with EBS resides in ITFR 5, “Retrieve SMS Item Master data from authoritative source (e.g. EBS) to support the SMS Requirement.” EBS is only one of 12 systems from which SMS retrieves data for ITFR 5. While the SMS side of the interface can be built, the external EBS interface is not available as planned. Figure 5 shows the traceability to other element types in the Requirements Repository that may be indirectly affected by a change to

ITFR 5.

Measure of Performance 13 is affected by the partial failure of ITFR 5; that the data transfer (including the

Figure 5: Requirements Affected by the EBS Interface through Failed Interface in ITFR 5

UNCLASSIFIED 10

EBS interface) should take place in less than three seconds. Information Asset 11, the SMS Item Master, is also affected, as is Business Process Activity 17, the planner getting cataloguing data.

Business Process Activity 17 similarly impacts Scenario 11, Maintenance requirements for the unplanned SMS requirement. The analytical trace of impacted element types continues throughout the Requirements Repository to impact Deployable Capability (DC) 11.

Once the trace of the Requirements Repository is complete, it is determined that the element types shown in Figure 5 are impacted by the missing EBS interface. The significance of the impact varies depending on the details of the element type. Other element types in the Requirements Repository are not affected by the EBS interface failure.

For this example, the completion of the Requirements Analysis determined the following:

1. Program capability is reduced by the failure to interface with EBS.

2. Initiative capabilities can be provided without the EBS interface if the EBS data is available, but without EBS data the SMS initiative capabilities are significantly degraded.

3. The measure of performance of three seconds for data transfer cannot be achieved until the modification to EBS is complete, reducing the overall benefit of the initiative, but with an alternate data source the initiative can still meet the user need.

Schedule Analysis

The SMS program can be executed against the current tasks in the IMS. While the EBS interface does not exist during Release 2 test and deployment, the testing tasks can be performed by simulation without fundamental changes to the testing task. Operational testing cannot be completed until the EBS upgrade comes on line. Release 2 program functions can be effectively produced and delivered, but the capability is degraded due to the lack of the EBS interface.

Figure 6 shows that the build and test of SMS does not change the existing schedule tasks in function, timing, or duration for the M-Implementation Plan. The SMS interface with EBS, Figure 6: Only Release 2 Build and Test Are Impacted, but Not Delayed

ID Task Name Duration Start Finish

808 MROi Development and Fielding Phase 1937 days Wed 8/7/13 Thu 1/7/21

809 Program Management (881C-K.1.3) 1937 days Wed 8/7/13 Thu 1/7/21

907 Award Activities 15 days Wed 2/18/15 Tue 3/10/15

927 Post Award Activities 26 days Tue 3/10/15 Wed 4/15/15

940 Release 1 - Standardized, Auditable, Enterprise V372 days Wed 3/4/15 Thu 8/4/16

983 Release 2 - Integrated Engineering Support and 786 days Wed 9/23/15 Wed 9/26/18

984 Integrated Engineering Support and Supply Support for Mx Activities DOT_LPF Change Management (R2)

20 days Wed 9/23/15 Tue 10/20/15

988 Release 2 - Build and Test (Integrated Enginee 214 days Wed 9/23/15 Mon 7/18/16

989 Pre build 9 days Wed 9/23/15 Mon 10/5/15

993 Build 135 days Tue 10/6/15 Mon 4/11/16

999 Dev Test 70 days Tue 4/12/16 Mon 7/18/16

1009 Release 2 - Field Test (Integrated Engineering and Supply)

10 days Tue 7/19/16 Mon 8/1/16

1019 Release 2 - Deploy (Integrated Engineering and Supply)

560 days Thu 8/4/16 Wed 9/26/18

1037 Release 3 - Optimized Mx Supportability 618 days Tue 6/21/16 Thu 11/1/18

1093 Release 4 - Agile and Consistent Mx Decision 551 days? Mon 5/8/17 Mon 6/17/19

1153 MROi Operations and Support Phase 2923 daysWed 11/4/15 Fri 1/15/27

Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug 2nd Quarter 3rd Quarter 4th Quarter 1st Quarter 2nd Quarter 3rd Quarter 4th Quarter 1st Quarter interface is not initially able to provide desired data.

Build, Test and Deploy can be accomplished to the planned standard, but the external interface is not initially able to provide desired data.

UNCLASSIFIED 11

however, is not functional for 18 months, thus delaying the functionality in the DOTMLPF-P Implementation Plan in Figure 7.

Figure 7: EBS Delays Capability Delivered by M-Solution

In order to retain functionality in the DOTMLPF-P Plan to meet the user need, some of the program benefit of reduced manpower will be delayed. By retaining sufficient manpower to perform the SMS / EBS interface, both the M-Implementation and DOTMLPF-P Implementation Plans can be executed as planned as shown in Figure 8.

Figure 8: DOTMLPF-P Milestones are aligned for successful execution

Cost Analysis

Based on the Schedule Analysis the tasks, task durations, and resources in the schedule have not changed. Therefore there are no changes to the program cost of the SMS program being executed according to the current plan as shown in Figure 9.

Figure 9: Current Program Status Quo Execution Has Same Estimated Cost

DOTMLPF‐P Implementation Plan

M‐Implementation Plan

IOCMS C FOC

M‐IOC M‐FOC

MS B

M‐MS CM‐MS B Policy Changes Training

Doctrinal Change

Base Year Estimate FY14 FY15 FY16 FY17 FY18 FY19‐27 Total

Baseline BY14 1,455,592$ 22,408,500$ 28,554,753$ 26,952,053$ 24,197,532$ 95,095,387$ 198,663,817$

Status Quo BY14 1,455,592$ 22,408,500$ 28,554,753$ 26,952,053$ 24,197,532$ 95,095,387$ 198,663,817$

Difference BY14 ‐$ ‐$ ‐$ ‐$ ‐$ ‐$ ‐$

UNCLASSIFIED 12

Risk Analysis

The Status Quo execution of the SMS program has one new risk and one increased risk associated with the triggering event. The new risk is the risk associated with the testing of the EBS interface which will now have to be simulated until the EBS system is available. There is the risk that the simulation does not accurately represent the EBS system as it is ultimately delivered and that the interface defects may not be recognized until very late in the SMS Program.

The risk that the EBS program may be further delayed must also be reevaluated. While the potential delay of the EBS update program was already identified as a risk, the likelihood of further delays has increased based on the triggering event. The risk register is updated for both of these risks.

Benefits Analysis

The SMS program promised to eliminate several FTEs in the current support structure, as well as identifying the value of responsive and accurate maintenance projections. Both of these benefits are reduced by the lack of an automated interface with EBS. By revisiting the original benefits estimates, the team generated new benefits estimates, shown in Table 3, with a differential reduction in benefits of $398,150. Table 4 shows the recalculation of the ROI accounting for the revised benefits.

Benefit Description Benefit Change

Reduce Manpower Savings

1 FTE must be retained to perform manual interface with EBS system

Reduction of $270,000

Slower Transfer Times

During the 18 month delay in functionality:

Requires 40 hours of overtime per month to compensate for the slower performance, and

Three percent of work planning activities are impacted with up to a two hour delay

Reduction of:

$72,000

$56,150

Table 3: EBS Benefits Analysis Based on the Status Quo

Costs Benefits ROI

Table 4: Status Quo ROI against the Baseline

Notify Senior Leadership

Senior Leadership is notified of the results of the Status Quo Analysis using the accumulated information. A summary slide such as Figure 10 is used to brief the Status Quo Analysis.

UNCLASSIFIED 13

Figure 10: Presentation of the Status Quo for the SMS Example

Step 3

Brainstorm Alternatives

The SMS Impact Analysis Team performs classical brainstorming activities, where all ideas are considered and none are evaluated or eliminated until the full set of alternatives is identified.

Figure 11 shows some of the alternatives developed by the team.

Potential Alternative Examples

1. Reduce requirements – eliminate the EBS interface requirement and perform the data entry of material lag time manually

2. Reduce requirements – eliminate the 3-second requirement and allow for a longer execution time

3. Substitute DOT_LPF-P capabilities for M capabilities – perform data entry manually

4. Design alternatives – move the problematic requirement from Release 2 to Release 3 or 4

5. Design alternatives – Use commercial translation software to perform data transfer. Allow for longer response times

6. Add time and funds – Lengthen development until the EBS system can support the interface

7. Add time and funds – Build an alternative interface that matches the current EBS system

8. Add time and funds – Defer IOC/FOC

9. Reduce requirements – Eliminate interface capability

Figure 11: Potential Alternative Examples

Status Quo – Current Program Status after EBS Delay

Value to the Air Force ‐‐ ROI 3.107:1

DOTMLPF‐P Effects:

• Use manual workarounds

• Interface will be finally enabled by the EBS upgrade program

Capability:

• SMS functionality is complete

• EBS interface capability is not functional until EBS upgrades are complete

• User must provide work around for EBS interface

Cost 121k 785k 1,382k 837k 411k 386k 125k 85k 1,200k

2013 2014 2015 2016 2017 2018 2019 2020 2021-50FY

MS B IOCMS C FOC

Risk:

1. Added risk of additional EBS upgrade delays.

2. Test SMS / EBS interface by simulation

No Change to Current Initiative Schedule for either the Materiel Program or the DOTMLPF‐P Plan

Program and Initiative Cost is unchanged

Costs Benefits ROI

UNCLASSIFIED 14

Filter Alternatives

Once the brainstorming is complete the alternatives were reviewed based on criteria in Figure 12. The rationale for excluding the alternative was recorded and will be reported to the decision-making authority. Table 5 exhibits an appropriate structure of reporting alternatives removed from consideration.

Alternative Identifier Description Reason for Elimination

1. Remove requirement The requirement for an automated interface with the EBS system will be eliminated

This is the same as or inferior to the Status Quo alternative

2. Eliminate the 3-second requirement

Allow for longer interface times

Incorporated in other alternatives.

Not a standalone solution

3. Lengthen development to match EBS availability

Delay SMS program until EBS interface is available

Unacceptable, inferior to the Status Quo:

a. Delays other capabilities

b. Risk of additional EBS delays or cancelation

4. Build interface to existing EBS Build an interface to the existing EBS system and transition to the new interface when EBS is updated

Unacceptable:

High cost, low benefit; Use of the initial interface will only apply for a short term, and should be replaced before SMS reaches full functionality

5. Defer IOC/FOC Delay program to respond to EBS delays

Same as #3 above

6. Eliminate interface requirement entirely

Remove requirement for SMS to use EBS data

Unacceptable:

Removes a significant capability from the SMS program, and does not meet the initiative need

Table 5: Alternatives Excluded from Consideration

Eliminate alternatives that are:

Redundant Unachievable Violate LRP Ruled out by Senior

Leadership

Figure 12: Eliminate Alternatives

UNCLASSIFIED 15

SMS Alternative to Perform the EBS interface manually

One of the SMS alternatives is to transfer Electronic Business System (EBS) interface from the M-Solution to a manual performance. The changes to the program plan in Table 6 implement this alternative.

Approved Initiative

Artifact Change

Requirements Repository Eliminate the EBS interface from ITFR 5 since that will be performed manually. The 3-second interface requirement will not apply to the manual process.

Update the IMS and resource loading of the IMS

Eliminate the effort associated with developing and testing the automated interface. Incorporate associated risk mitigation activities into the IMS.

Add to the DOTMLPF-P Implementation Plan

Add 1 FTE as a Personnel Capability necessary to support the manual interface.

Update the Business Reference Model

Update the Business Reference Model (BRM) as necessary to account for the manual data interface between SMS and EBS.

Table 6: Changes Due to an Alternative

Step 4 and Step 5

Once these changes are applied to the program and initiative artifacts, the updated artifacts are evaluated through the same procedures as the Status Quo. The variance against the approved requirements is identified, as is the schedule and cost variance. Overall ROI is developed, and the impact of the alternative(s) is reported using the same format as for the Status Quo.

Appendix B – Acronym List

Acronym/Term Definition ADM Acquisition Decision Memorandum AF Air Force BRM Business Reference Model BY Base Year COA Course of Action COTS Commercial Off-The-Shelf DoD Department of Defense DoDD Department of Defense Directives DOT_LPF-P Doctrine, Organization, Training, Leadership, Personnel, Facility, and

Policy DOTMLPF-P Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facility, and Policy DRM Data Reference Model EBS Electronic Business System FM Financial Management FTE Full Time Equivalent FY Fiscal Year GOTS Government Off-The-Shelf IMS Integrated Master Schedule IPO Integrated Program Office IPT Integrated Project Team IT Information Technology ITIL Information Technology Infrastructure Library M Materiel OMB Office of Management and Budget PM Program Manager PRM Performance Reference Model PMO Project Management Office ROI Return on Investment SDDP Service Development and Delivery Process SME Subject Matter Expert SMS Supply Maintenance System TTP Tactics, Techniques and Procedures WBS Work Breakdown Structure

UNCLASSIFIED 17

Appendix C – Glossary

Term Definition Bounded User Requirements

The complete set of requirements that describe the M-Solution; it is comprised of the SRM, DRM and M-Implementation Plan.

Business Reference Model

A function-driven framework used to describe the lines of business and sub-functions performed by the federal government independent of the agencies that perform them. IT investments are mapped to the BRM to identify collaboration opportunities. (Source: OMB Circular A-11, Section 53)

Capability The ability to achieve a desired effect under specified standards and conditions through combinations of means and ways across the full spectrum of DOTMLPF to perform a set of tasks to execute a specified Course of Action (COA). It is defined by an operational user and expressed in broad operational terms in the format of an initial capabilities document or a DOTMLPF change recommendation.

Configuration Control

Activities concerned with ensuring that only authorized and identifiable Configuration Items (CIs) are recorded throughout their lifecycles and that no CI is added, modified, replaced or removed without appropriate controlling documentation, e.g. an approved Request for Change.

Configuration Control also includes controlling access to the information held on CIs. (Source: ITIL)

Course Of Action

A planning and decision process that culminates in a sponsor decision. It principally refers to the decision whether to proceed with development of one or more prospective M-Solutions.

Data Reference Model

A framework used to promote the common identification, use and appropriate sharing of data/information across the federal government. It provides standards and guidelines to help agencies structure, categorize, exchange and manage their data to improve the ability of government to perform cross-agency information sharing. (Source: OMB Circular A-11, Section 53)

Deployable Capability

The smallest reasonable increment of useful product that can be provided to the end user.

DOTMLPF-P

Capability

A specific action or portion of a DOTMLPF-P solution that, when completed or deployed, has standalone value to the Warfighter. A collection of DOTMLPF-P capabilities, within the context of the SDDP, solves the end user’s problem or satisfies the end user’s need.

Information Technology

The Clinger-Cohen Act of 1996 (40 U.S. Code, § 11101) defines information technology as: The term "information technology" - (A) with respect to an executive agency means any equipment or interconnected system or subsystem of equipment, used in the automatic acquisition, storage, analysis, evaluation, manipulation, management, movement, control, display, switching, interchange, transmission, or reception of data

UNCLASSIFIED 18

or information by the executive agency,” And includes computers, ancillary equipment, software, firmware and similar procedures, services (including support services), and related resources.

Integrated Master Schedule

A time-based schedule containing the networked, detailed tasks necessary to ensure successful program/contract execution. The IMS is traceable to the integrated master plan, the contract Work Breakdown Structure, and the statement of work. The IMS is used to verify attainability of contract objectives, to evaluate progress toward meeting program objectives, and to integrate the program schedule activities with all related components.

Materiel Development Decision

A formal entry point into the acquisition process and is mandatory for all programs. It identifies a gap in capability and develops requirements to fill that gap. The decision is documented in the Acquisition Decision Memorandum (ADM).

Materiel Implementation Plan (M- Implementation Plan)

The plan for developing/acquiring and delivering the M-Solution. The plan should include all activities needed across skill sets (e.g., architecture, system engineering, acquisition, funding, contracting, governance, testing and evaluation, and compliance).

Materiel Solution (M- Solution)

Correction of a deficiency, satisfaction of a capability gap, or incorporation of new technology that results in the development, acquisition, procurement or fielding of a new item (including ships, tanks, self-propelled weapons, aircraft, etc., and related software, spares, repair parts and support equipment, but excluding real property, installations and utilities) necessary to equip, operate, maintain and support military activities without disruption as to its application for administrative or combat purposes.

(Source: DoDD 4630.05) Within the context of the SDDP, the M-Solution is focused on the acquisition or development of Information Technologies, specifically software.

Mission Capability

A Mission Capability is the second level of the WBS.

Performance Reference Model

A framework to measure the performance of major IT initiatives and their contribution to program performance. The PRM helps agencies to produce enhanced performance information; improve the alignment and better articulate the contribution of inputs, such as technology, to outputs and outcomes; and identify improvement opportunities that span traditional organizational boundaries.

Point Estimate An estimate that represents only one value from a range of possibilities. In most cost estimating applications, the relationship of the point estimate to the range of possibilities is unknown until after the risk analysis has been performed.

Program A formal Materiel acquisition with a program acquisition staff.

Project A Materiel acquisition that may or may not be large enough to be a program.

Return On A measurement of performance used to compare the effectiveness of

UNCLASSIFIED 19

Investment different investments. ROI is calculated by taking the benefit, or return, from an investment and dividing it by the cost of the investment. ROI is expressed as a percentage or a ratio.

Risk Mitigation A systematic reduction in the extent of exposure to a risk and/or the likelihood of its occurrence through active measures.

Risk Register A Risk Management tool commonly used in Project Management and organizational risk assessments. It acts as a central repository for all risks identified by the project or organization.

Schedule-Based Estimating

Cost estimating using a schedule as the basic Cost Element Structure

(CES).

Service Development and Deployment Process

The end-to-end process, by which capability-supporting mission processes are identified, described, developed and deployed.

Senior Leadership

Senior Leaders are the chartering members of the IPT/IPO to include the Sponsor, Functional Lead and PEO

Service Reference Model

A description of the functions and services that the M-Solution must deliver in order to solve the end user’s problem and/or satisfy the end user’s needs.

The SRM information must be at a sufficient level of completeness, consistency and detail to prepare a bounded user requirement sufficient to select a satisfactory COTS product or develop software code to deliver the M-Solution successfully.

Triggering Event An unplanned event with implications concerning the optimal program execution.

Work Breakdown Structure

An exhaustive, hierarchical tree structure of deliverables and activities that need to be performed to complete a project. A WBS is a common project management tool and the basis for much project planning.

Work Plan A charter or plan to perform specific work. Within the SDDP, it is often focused on completing a single Step of the SDDP and typically consists of:

tasks, resources, timelines, and milestones.

UNCLASSIFIED 20

Appendix D – References

Air Force Manual 33-402, Service Development and Delivery Process (SDDP)

TTP for Work Breakdown Structure

TTP for Integrated Master Schedule

TTP for Integrating Risk into Program Plans

TTP for Schedule-Based Estimating

TTP for Mission Applications Requirements Capture and Traceability

The referenced TTPs are available at the AF Executive Expertise Team SharePoint repository at:

https://cs1.eis.af.mil/sites/AFExec_UST/SitePages/Home.aspx