Technical_Exhibit_-_EVMS_at_PIADC_(v1)_(20170130).pdf

Technical Exhibit - Earned Value Management System (EVMS) Description and Guidelines for the Tailored Application of EVM at PIADC

Introduction

The Department of Homeland Security (DHS) at the Plum Island Animal Disease Center (PIADC) is committed to implementing project management practices in order to effectively and efficiently manage the cost and schedule of work. To meet this commitment, the contractor will be required to use Earned Value Management (EVM) to plan and monitor their cost and schedule performance. EVM requires the development of an integrated cost and schedule baseline that provides visibility into how, when and where resources are allocated over the course of the performance period. Typically, baselines are established for each 12 month period that corresponds to the contract period of performance. The contractor’s award fee is connected in large part to their ability to execute the baseline plan, in addition to other factors.

Purpose and Scope

This system description has been developed to capture facility-specific processes, guidelines and practices that shall be followed by the managing contractor of the Center Wide Support Program (CWSP) and their subcontractors (to the extent and degree practical) at PIADC.

The contractor will be expected to develop and implement a robust, tailored project management approach for operations, maintenance and research support activities covered under the contract statement of work (SOW) using EVM techniques to monitor and measure cost and schedule performance. This requirement is intended to motivate the contractor toward careful planning and execution of all program work against a validated cost/schedule baseline that utilizes key performance indicators (metrics) that will be tied to award fee, while also incorporating core principles of quality (ISO 9001) and risk management.

To the extent possible and practical, all work within the SOW should be viewed as “project work” and shall be planned in a manner that allows earned value credit-taking and reporting against the baseline. A discussion of how earned value can be implemented in a services environment (i.e., where level of effort (LOE) type work may be dominant) is presented later in this document.

The contractor must develop and submit an earned value management system description that provides greater detail about their schedule development & control, cost management (budgeting & cost accounting), EV credit-taking, change control and reporting process (complete with thresholds, variance analysis and forecasting processes) as part of their project management plan.

Background

Since 2010 the O&M contractor at PIADC has been required to implement an EVMS that is consistent with the tenets of the following standards:

• “Earned Value Management”, ANSI/EIA 748-C (2013)

• “Practice Standard for Earned Value Management”, 2nd Edition , PMI (2011)

• “A Guide to the Project Management Body of Knowledge (PMBOK® Guide)”, 5th

Edition, PMI (2013)

• “Earned Value Management Guidance”, Version 1.1, Department of Homeland Security

(February 2007)

• “Integrated Program Management Report (IPMR)” Data Item Description (DID), DI-

MGMT-81861A (Formats 1 – 7), Department of Defense (DoD) (2016)“Integrated Program Management Report (IPMR)”, DI-MGMT-81861A (Formats 1 – 7), Department of Defense (DoD) (2015)

These standards continue to be applicable for how cost and schedule performance will be planned, measured and reported by the CWSP contractor. A graded approach has been used to tailor these standards for application in a services environment. Earned value management has provided visibility into how planning decisions are made, where resources are allocated and specifies the timeframes for when money is to be spent for both project work and level of effort (LOE) services. The government has been able to review and validate the contractor’s fiscal year work plans with a much greater degree of confidence than if earned value management were not being used. Reporting and tracking of variances has also been enhanced: without EVM the contractor would not have been able resolve a cost or schedule variance down to the level it is currently shown.

Performance or quality metrics have been established to represent the “earned value” of work that would normally be considered level of effort. For example, several late ferry departures would be converted into partial EV credit and therefore a negative schedule variance for the transportation control account, which would typically be viewed as a support activity that earns the same amount of credit as was planned in the PMB. Metrics are developed jointly between DHS and the contractor, keeping in mind that the contractor’s award fee is linked to how much value is earned. The design of metrics has been the key to transforming LOE from noise into signal. This transparency has allowed for a more thorough understanding of the root causes of variances and supports development of more effective corrective actions. Forecasting of cost and schedule performance has also vastly improved when utilizing EVM because trends in spending or productivity can be estimated with greater precision.

EVM has been integrated into an overall strategic planning and execution cycle that involves several components. These components relate scope, cost/schedule and performance measures to each other throughout the fiscal year. The flowchart below depicts how some of these performance management processes interact before, during and after a Fiscal Year (FY) cycle.

Acronyms used in the process flowchart are defined below:

• AFP - Award Fee Plan*

• DHS - Dept. of Homeland Security

• FY - Fiscal Year

• IBR - Integrated Baseline Review*

• JSR - Joint Surveillance Review*

• PG - Planning Guidance*

• PMB - Performance Measurement Baseline*

• SOW - Statement of Work

*These items are discussed and defined elsewhere in the SOW.

The Performance Measurement Baseline (PMB)

The PMB includes all information necessary to measure performance using an earned value management framework, including metrics for LOE type work. It includes the following components:

• Work Breakdown Structure (WBS) and WBS Dictionary

• Integrated Master Schedule (IMS)

• Time-Phased Cost Plan (by control account by month)

The PMB shall be developed around a complete and comprehensive work breakdown structure that captures all approved scope in the current SOW. The WBS will be developed using current accepted practices for organizing scope into a framework of related deliverables, such as Work Packages and Control Accounts (CA), all of which shall be identified using a hierarchical numbering system. Other common practices may be employed in creating the WBS (such as use of Planning Packages) provided these techniques are fully documented in the contractor’s system description (project plan) and agreed to by DHS. The contractor’s project management plan will contain additional information that supplements the PMB, such as:

• Responsibility Assignment Matrix (RAM) - Table that relates control account managers (CAMs) to specific elements of the WBS.

• Work Authorization Documents (WADs) – Summary descriptions of each major control account that provides total dollars with breakouts showing standard hours, overtime hours, indirect costs, travel, subcontract costs, material costs, G&A and ODCs. WADs are signed by the program manager and each CAM so that scope, cost and timeframe of work is clearly understood and agreed-upon.

• Performance/Quality Metrics – Table showing clear definitions for each metric that will be used for “earned value” credit-taking.

While there may be a tendency to overestimate costs or durations of work in the PMB during planning, in order to overstate earned value performance later, this practice is discouraged and ultimately counterproductive. The goal of estimating cost and schedule values is to create an authentic and realistic plan against which performance can be measured. An integrated baseline review (IBR) will be conducted to validate the PMB which will uncover anomalies, unrealistic estimates and disconnects between the planned work and the resources that have been assigned.

Monthly reviews of performance against the baseline also tend to uncover errors or distortions in the plan. Deliberate misrepresentations that are discovered in the PMB could lead to reduced fee (compensation) based on lack of baseline realism and integrity.

Furthermore, the PMB must include the integrated master schedule (IMS) which depicts all of the activities and milestones together with their predecessors, successors, typically as a Gantt chart with supporting detail. Resources shall be aligned with the IMS (or preferably within the IMS), so that time phased budgets (“planned values”) can be established for each control account for each month. Cost and schedule performance of the contractor is measured against the PMB, which is essentially comprised of the integrated WBS (scope) and IMS (schedule and cost). If the IMS does not contain all of the time-phased budgets then this must be provided elsewhere to show total resources planned by month. Once the three parameters of scope, schedule and cost are approved in the form of the baseline (PMB), the three corresponding earned value management parameters can be measured during project execution.

EVM Parameters

The Earned Value Management (EVM) technique allows project performance to be monitored and controlled using three basic parameters:

• Planned Value (PV) - The planned budget for work based on estimates or historical information. PVs can be assigned at various levels of the project (the most accurate approach is to determine planned values for each activity, but this is not always practical). Each PV is locked in-place with the schedule (or timeframe) for performing that scope of work. In this way, a monthly cost plan can be determined in advance by summing the PVs for that month. The cumulative distribution of all PVs creates an S- Curve which is often referred to as the "cost baseline." The sum of all PVs will equal the Budget at Completion (BAC) for the project. It should be noted that the BAC does not include dollars related to seller's fee (profit) or management reserve. PVs are used primarily to measure schedule performance (against what was actually "earned").

• Earned Value (EV) - The value of work that was actually performed during a reporting period. It is commonly calculated by multiplying the percent complete times the planned value for that portion of work being measured (or EV = %Complete x PV). For example, if a work item has a PV of $10K and 90% of that work was completed, then only $9K was earned. This becomes a critically important measure since it represents the amount of planned scope that was completed (regardless of when it was completed or how much was spent). EV is always used in determining cost performance and schedule performance, though both of these are usually independent variables. EV is the common thread that relates to "how much scope" was completed for the money spent or time elapsed. Different techniques can be used to take EV credit including weighted milestones, discrete effort, EV credit rules, or level of effort (LOE). The type of earned value methods to be applied would be established during planning.

As mentioned earlier, metrics should be developed for work that is largely LOE or based on providing ongoing services (since discrete deliverables are not typically generated for this type of work which therefore diminishes the meaning of what is truly “earned”).

Development of metrics must carefully consider the following:

• What aspect of the service can be readily quantified?

• Do these aspects have meaning and are they significant?

• How much value should be placed on the metric and how will it be weighted against the other metrics in that area?

• What range should be used in determining full or partial credit?

• Is the high-end of the credit scale for the metric achievable, realistic and challenging (difficult)?

Metrics shall be traceable from the top of the organization to the bottom, where performance (work) takes place: strategic goals are accomplished via critical outcomes, which in turn are aligned with scope using the WBS control accounts. Metric alignment should be clear and their content and value may be adjusted during the course of the performance period in order calibrate how scores are translated into cost or schedule variances. However, since metrics are part of the PMB, these adjustments are expected to be incremental, so as not to produce large swings in performance. Metrics can be more dramatically revised when a new PMB is developed.

• Actual Cost (AC) - The money spent to perform the work that was completed. These are considered "sunk costs" and are typically retrieved after the fact through the organization's cost accounting system. By subtracting AC from EV, the cost variance is determined. Note - this is different and more reliable than simply subtracting AC from PV (which should NOT be done when using EVM).

Cost and Schedule Variances

The cost and schedule variances provide visibility into project performance in terms of dollars.

This information is accumulated monthly for each control account and is summarized directly through the WBS. The two variances are described below:

• Cost Variance (CV) is calculated by subtracting AC from EV (CV = EV - AC). A positive cost variance indicates that the completed work cost less than the value of the work that was earned or planned. Conversely, a negative variance indicates that more money was spent than was earned, for the work that was completed.

• Schedule Variance (SV) is calculated by subtracting PV from EV, providing a dollar value for schedule variance (SV = EV – PV). A positive schedule variance means that more earned value was taken than planned, which is favorable (ahead of schedule). A negative schedule variance means that less value was earned than planned, for that reporting period, which is an unfavorable condition (behind schedule). Care must be exercised in drawing inferences from either favorable or unfavorable schedule variances because critical path activities may or may not be represented accurately when using an aggregated SV for the entire project. Crosschecks must be made using the schedule to determine the true status of activities.

Cost and Schedule Performance Indices

• Cost Performance Index (CPI) - A measure of cost efficiency expressed as a unit-less fraction or ratio of what was earned versus what was spent (CPI = EV / AC). When actual costs are greater than the earned value then CPI is less than 1, which indicates that more money was spent to do the planned work. A CPI = 0.95 means that for every dollar spent, only 95 cents of work was accomplished. This trend will result in a cost "overrun" situation if not corrected. Conversely, a CPI greater than 1 implies that more work is being accomplished for less money. While this is considered favorable, it may also indicate that the baseline was not representing the planned work accurately. Performance that is considered to be "On Budget" would have a CPI equal to 1. This measure is important for cost plus award fee (CPAF) contract management because it can be used to highlight those areas of scope that cost more than planned (even if schedule performance was satisfactory).

• Schedule Performance Index (SPI) - A measure of schedule efficiency expressed as a unit-less fraction or ratio of what was earned versus what was planned (SPI = EV / PV).

When earned value is less than what was planned then SPI is less than 1, which indicates that some fraction of work was not completed on-time. An SPI = 0.95 means that only 95 percent of planned work was accomplished "on-time" (there would still be 5% of planned work remaining). This means that work is not being completed when it was planned for completion (regardless of how much it cost or what was spent). This measure is important for CPAF contract management because it can be used to highlight those areas of scope that were not completed when planned (even if cost performance was satisfactory).

Analysis of performance measurement data (CV, SV, CPI and SPI) will allow the contractor to identify and document the cost, schedule, and work scope conditions that may require management attention, assess the impact of these conditions on the baseline and future work, and develop and implement corrective actions as necessary.

EVM in a Services Environment

Applying EVM in a services environment is challenging because the earned value of strictly LOE work only documents the passage of time and not actual project progress (that is, the planned value equals the earned value as soon as the time period elapses). For this reason, LOE is often considered the least desirable type of work to measure using EVM. However, the earned value of LOE work can become more meaningful through the use of metrics.

Historically, more than 50% of the Operations and Maintenance (O&M) activities at PIADC has been comprised of LOE work. Much of this LOE work has been effectively converted into more meaningful “planned values” by using metrics during the reporting period (monthly). Much of the ongoing O&M work has been “projectized” so that within each fiscal year cycle the work that would be considered LOE is broken down into measureable segments that relate to earned value performance. The contractor will be expected to develop and expand their utilization of metrics so that the effective percentage of LOE work is reduced and specific outcomes are attached to as much of the time-phased budget as possible.

Earned Value Metrics

Metrics will be “Contractor-proposed, Government-approved” as part of a broader framework which supports critical outcomes (critical outcomes will be “Government-provided”). An example of this alignment is presented in Figure 1 below (i.e., this example is subject to change or modification and is for demonstration purposes only):

Figure 1 – Sample Metrics

Goal 1 - Provide Outstanding Service to Science Mission Critical Outcome 1.1

The availability and reliability of critical infrastructure systems and components are maintained to provide uninterrupted services to the PIADC research mission

Measure 1.1.1 Animal room cycle times are optimized to support maintenance with minimal interruption to research operations

EV Credit (Score) Metrics (Thresholds) 100% All animal room PMs are performed as scheduled (during the month) without interruption to planned research utilization and SPI = 1 75% 1 to 3 animal room PMs are delayed or extended (during the month) without interruption to planned research utilization and SPI ≥ 0.95 50% More than 3 animal room PMs are delayed or extended (during the month) without interruption to planned research utilization and SPI ≥ 0.9 25% More than 3 PMs are delayed or extended (during the month) with or without interruption to planned research utilization and SPI ≥ 0.8 0% Significant interruption to planned research utilization of animal rooms due to execution, planning or performance issues within the Contractor’s control or SPI ≤ 0.75

Notes:

1. All metrics above assume that research operations are impacted due to issues related to the Contractor’s control of work

Upon receipt of the Government-provided critical outcomes, the Contractor will develop the supporting measures, metrics and notes to clarify earned value credit-taking rules (as in the example shown above) with concurrence and approval of the Government, as part of each annual PMB submittal. The Contractor is expected to provide supporting metrics that are specific, measurable, achievable, realistic, timely and properly aligned with PIADC mission objectives, mapped to critical outcomes and traceable to relevant work breakdown structure (WBS) control accounts. An example of this kind of mapping is provided in Figure 2 below:

Figure 2 – Goals, COs and Measures Mapped to WBS

Award Fee Plan (AFP) Measures

The AFP is used to provide incentives in specific areas of CWSP performance: core services, projects, quality management, resiliency and sustainability for each option year. The base year may establish slightly different award fee incentives designed to “stand-up” the Contractor’s management systems in a timely manner, while also demonstrating that these systems are properly implemented and effective.

A significant portion of “core services” award fee will be linked to CPI and SPI so that the contractor has a clear incentive to meet or exceed baseline targets for cost and schedule. This in turn will optimize the utilization of resources by the contractor. The contractor is expected to establish baseline targets that are realistic and achievable without including unnecessary costs for equipment, materials or personnel, especially if work can be accomplished using a leaner resource model. Any attempt made by the contractor to “pad” or artificially increase estimates (in order to make subsequent performance look better) will be discovered and corrected during validation of the baseline. The AFP will reward performance indices above 1 and create disincentives for cost and schedule performance that is less than 1.

The actual numeric scales of CPI and SPI that relate to award fee scores will be developed jointly between the contractor and DHS prior to baseline validation. The agreed-upon scales will be based, in part, on the maturity of the contractor’s EVMS, realism of the baseline and the

Pr og ra m a nd

P ro je ct

M an ag em en t

Q ua lit y

M an ag em en t

Bu si ne ss

M an ag em en t &

Su pp or t

An im al C ar e, U se

H an dl in g

La bo ra to ry a nd B io co nt ai nm en t Se rv ic es

U til iti es a nd

F ac ili tie s S ys te m s

In fo rm at io n

Te ch no lo gy a nd

Co m m un ic at io n

Sy st em s

Sa fe ty , H ea lth

, E nv iro nm en ta l, an d Em er ge nc y

Se rv ic es

T ra ns po rt at io n

B ui ld in gs , R oa ds a nd

G ro un ds

H os pi ta lit y an d

St af f S er vi ce s

ID

IQ

T as k O rd er s / P la nn in g

Pa ck ag es

1.1.1 Animal room cycle times are optimized to

support maintenance with minimal interruption to research operations

GOAL Critical Outcome (Government-Specified)

Measure (Contractor-Proposed and Government-

Approved)

WBS / CONTROL ACCOUNT TITLE

- P ro vi de O ut st an di ng S er vi ce to

S ci en ce

M is si on

1.1 The availability and

reliability of critical infrastructure systems and components are maintained to provide uninterrupted services to the PIADC research mission anticipated difficulty of the contractor to achieve the highest score. The highest score for any award fee measure should not be achieved easily. That is, the contractor will be expected to demonstrate a high degree of efficiency, ingenuity and disciplined effort in order to score an “Excellent” for all measures contained in the AFP.

Award fee for “core services” will be based on a significant portion of LOE work and therefore rely heavily on CPI and SPI. Clear rules for taking earned value credit are essential in order for this process to work properly (i.e., motivate the Contractor to meet or beat their baseline targets).

Some portion of award fee may be linked to performance in other areas, such as quality management, resiliency and sustainability. The allocation of award fee to each area, their definitions and the processes involved in establishing and using the AFP are discussed in a separate TE (TBD).

Reporting and Forecasting

The Contractor Performance Report (CPR) is used to collect and present earned value performance data to DHS each month utilizing Formats 1, 2 and 5 as defined by the Department of Defense (DoD) Integrated Program Management Report (IPMR):

• Format 1 - Work Breakdown Structure (form 2734/1)

• Format 2 – Organizational Categories (form 2734/2)

• Format 5 - Explanations and Problem Analyses (form 2734/5)

In addition, the Contractor must be prepared each month to provide, review and discuss the following:

• Current staffing levels (or “head count”) compared to proposed staffing in the Human Resource Management Plan (HRMP) and/or latest baseline

• Definition and status of “metrics”, progress against key milestones and award fee measures, as applicable.

Monthly information requirements may be, added, removed or tailored at the request of DHS or through mutual agreement and concurrence between the Contractor and Government.

CPR use is intended to be consistent with IPMR guidance (reference DI-MGMT-81861A), however, exceptions will be considered. The organization of data, grouping of WBS element by Control Account, and level of detail may be revised, if appropriate. The Contractor and DHS should view the CPR as a means to provide insight into spending, progress and forecasting therefore any revisions to the reporting format(s) should preserve this intent. Any changes to the standard CPR templates must be reviewed and approved by DHS prior to use. In addition, the Contractor must be prepared each month to review the definition and status of “metrics”, progress against key milestones and award fee measures, as applicable.

Management Reviews

Program Management Review (PMR) - The contractor shall conduct a PMR with the Government on a quarterly basis to determine how effectively the overall program is being managed in accordance with the provisions of the contract (e.g. quality, cost and schedule performance project management, environmental management, etc.). The management review requirements of ISO 9001 shall be used as the basis for planning and conducting the PMRs.

Integrated Baseline Review (IBR) – A review of the contractor’s integrated Performance Measurement Baseline (PMB) will occur annually to coincide with each new PMB that is developed to describe be the work planned for the upcoming fiscal year. The IBR focuses on the realism of scope, cost and schedule in the PMB and the appropriateness of their management policies, processes, and the implementation of earned value and quality measurement methodologies.

Joint Surveillance Review (JSR) – PMRs and IBRs shall be complemented by Government-led Joint Surveillance Reviews (JSRs) conducted roughly midway during fiscal year in order to uncover any significant issues or inconsistencies with the plan or approach that has been adopted to execute the PMB. These reviews shall serve to drive continuous improvement in project management and sustainable performance methodologies.

Planning Guidance (PG)

Roughly 3 months prior to the start of each fiscal year the DHS will provide guidance to the Contractor that shall be used in the development of their PMB. This “planning guidance” will be provided at roughly the same time that the AFP is prepared so that the Contractor can align their efforts accordingly. Some of the planning guidance may relate to complex-wide improvements, initiatives and/or projects that are considered to be mission-critical. Planning guidance will also include an anticipated contract funding profile that should be used as a basis for the resource allocations in the baseline. The PG will not include facility upgrades, maintenance, and repairs that are well within the routine responsibilities and control of the CWSP contractor (e.g. roof maintenance, autoclave maintenance, etc.).

The PG will direct the contractor to include all work from the SOW in the latest PMB, as well as any new work, as this becomes better defined or as priorities change. Contractor management and technical staff should expect to meet with the appropriate DHS technical leads during each planning cycle to assure a clear and mutual understanding of the work as definition of scope evolves.

Change Control

The contractor will need to develop appropriate processes and procedures that facilitate changes the PMB. Changes may be required due to new or evolving scope, based on direction by DHS or due to changes in conditions, assumptions or constraints. The contractor’s change control processes must be aligned with their EVMS such that any changes to the contract SOW are coordinated to the scope, cost and schedule in the PMB and can be adequately measured using

EVM. All changes should be balanced with consideration of available resources, adherence to quality objectives and reduction of risk (both programmatic risk and those risks related to safety, environment and health).

Flow Down to Subcontractors

The contractor shall flow-down EVM and the reporting requirements to major or critical subcontractors. These subcontractors shall report performance information that the contractor will incorporate and integrate into its management system. The contractor shall review and assure the validity of all subcontractors reporting through regular surveillance and performance reviews. The contractor shall integrate subcontractor performance management information into the contractor's CPR, Integrated Master Schedule (IMS), IBRs and Program Management Reviews.

Exceptions

In general, fixed fee contract line items (FF CLINs) will not be subject to EVM reporting requirements. EVM is also generally not required for relatively small subcontracts involving T&M or LOE work, including Purchase Orders for supplies, equipment or materials that were not already included in the baseline. However, all purchases of equipment, material, supplies and service contracts should be integrated into the contractor’s cost/schedule baseline during planning since together these may result in significant aggregate cost within the CPAF contract, unless these are specifically excluded as part of a separate FF CLIN.

Risk Management

The contractor will be required to establish a process for identifying risks in all areas of the program (at a minimum: cost, schedule, scope, quality and ESH) and formulate effective responses based on a thorough analysis. Risk mitigation activities should be agreed upon by affected stakeholders (especially DHS) and reflected in the PMB during planning. During execution, risks should continue to be evaluated and discussed in the variance analysis report, monthly or as appropriate. The contractor will be expected to develop a Risk Management Plan (RMP) used to document procedures for managing the overall process and maintain a Risk Register, which will contain real-time, detailed information of each documented risk.

Undistributed Budget and Management Reserve

To the extent possible, the use of undistributed resources should be minimized in the baseline and stated clearly when and where these are used. Management Reserve (MR) should also be clearly identified and linked to risk mitigation activities related to baseline execution and it is expected that MR is supported by a documented analysis found in the contractor’s risk register.

EVMS Validation

The Contractor shall be prepared to self-declare conformance of their PM/EVMS against their Government-approved Project Management Plan three months after the contract transition period ends. The Government will commission an independent evaluation of the Contractor’s PM/EVMS plan to assure that the program meets Government’s requirements and objectives no later than the end of the base year.