DCMA14PointAssessmentSpec_final.pdf

Technical Specifications Earned Value (EV) DCMA 14-Point Assessment Software

General Information

1. General Description

The Defense Contract Management Agency (DCMA) contemplates establishing a firm fixed-price Blanket Purchase Agreement (BPA) for Earned Value (EV) DCMA 14-Point Assessment software licenses and maintenance. This acquisition will include a base year and four one year options.

2. Background

As the DoD’s Executive Agent for EVMS oversight, up to 500 EVM Specialists across the Agency are currently performing the DCMA “14 Point Assessment” on a monthly basis using Supplier schedules which are typically delivered in MS Project (majority), Primavera, or Open Plan format. Currently, there is no DCMA-wide software tool under configuration control which is capable of performing this often mandatory schedule assessment on any of the three common software platforms (MS Project, Primavera, or Open Plan).

3. Scope

The scope of this effort is to acquire a solution which is capable of performing the DCMA 14-Point Assessment using Microsoft Project files as input. All proposed software must comply with applicable Department of Defense (DoD) and local regulations and policies regarding DoD Information Security.

4. Software Specifications and Services (Minimum Requirements)

Solution must meet the following requirements:

• Produce the metrics required to satisfy the DCMA 14-Point Assessment.

o Metric Requirements: Reference Appendix B.

• Identify problem areas that may affect schedule quality.

• Must accept Microsoft Project 2010 files as input to produce the DCMA 14-

Point Assessment.

• Must be able to export results to Microsoft Excel 2010.

• Must have built-in feature to exclude Level of Effort (LOE) tasks from analysis

• Must have built-in feature to exclude Planning Package tasks from High Duration metric.

• Must have integrated Help feature which explains each DCMA metric

• Provide licenses for 125 users.

• Must run on a Windows 7 desktop computer.

5. Software Maintenance Services

The contractor shall provide software, licenses, version upgrades, patch upgrades, and phone support for all solutions described above for up to five years.

6. Software Delivery

Contractor will deliver software electronically.

7. WAWF Invoicing/Software Description Summary

In accordance with DFARS clause 252.232-7003, the contractor shall submit all invoices electronically. In addition, the contractor shall complete the DCMA document “Software Description Summary” (Appendix A) and attach it electronically within WAWF upon entering the invoice data. An invoice not including this document and accurate information will be rejected as incomplete.

8. Period of Performance

The period of performance shall be for one base year plus four one-year option periods.

9. Place of Performance

Software will be installed by DCMA technicians at various locations within the Agency.

10. Security Requirements

Not Applicable.

11. Post Award Conference/Periodic Progress Meetings

Progress meetings will be held by the COR or COTR to apprise the contractor of how the government views the contractor's performance and the contractor will apprise the Government of problems, if any, being experienced.

Appendix A - Software Description Summary

Instructions: DCMA loads this information into its system for tracking software. Please complete/update the information in Blue. At a minimum there should be one product description for each CLIN in the contract or manufacturer’s part number. POC for completion of this document is Denise Spillman Denise.Spillman@dcma.mil.

Contract: Contract# Order: Order #

Current Maintenance Start Date: MM/DD/YYYY End Date: MM/DD/YYYY

Product: Software Name 1

Manufacturer’s Software License SKU / Part Number: XXXXXX Manufacturer’s Software Maintenance SKU / Part Number: XXXXXX Includes the following:

Unabbreviated non-technical description of software purchase. If the software has components, then each component shall be listed.

License Constraint: XXXX users, XXXX devices, ...

Product: Software Name 2

Manufacturer’s Software License SKU / Part Number: XXXXXX Manufacturer’s Software Maintenance SKU / Part Number: XXXXXX Includes the following:

Unabbreviated non-technical description of software purchase. If the software has components, then each component shall be listed.

License Constraint: XXXX users, XXXX devices, ...

Schedule Analysis:

14 Point Assessment

APPENDIX B

Schedule Statistics

• Prior to performing the 14 point assessment, some basic schedule statistics must be calculated

• The statistics include:

– Total Tasks (lowest level discrete tasks)

– Complete Tasks (How many tasks have been completed?)

– Incomplete Tasks (How many tasks are not done yet?)

– Baseline Count (How many tasks were planned to be done by now?)

o A separate “BEI Baseline Count” is tracked as well which includes the number of tasks missing baseline start or finish dates in the schedule

– Relationship Count (How many FS, FF, SS, and SF relationships are there?)

• The statistics do not indicate anything about the technical structure of the IMS or the supplier’s ability to plan the work and work the plan

Useful to track maintenance in a growing/evolving schedule

Total Tasks

• How do I identify them?

• Tasks that are not any of the following:

• Summary Task or Subproject task

• An Earned Value Type of Level of Effort

• Tasks with a baseline duration of zero (or Milestones [Start / Finish])

Primavera • Summary tasks are often called “Level of Effort” in Activity Type by definition

• Designation of LOE in Earned Value Type could be “LE” or “Level of Effort EV”

• The column for baseline duration is called BL Project Duration

MS Project • There is a column for Summary with a Yes or No

• There is a column for Baseline Duration

• Talk with scheduler to determine the EV Method column in IMS. It should be a “Text” field

Open Plan • Summary tasks are called “Subproject” in Activity Type

• A task can be considered a milestone and have a duration more than 0 or 1 days

• The column for baseline duration is called Original Duration

“Total Tasks” are the lowest level discrete tasks

Complete Tasks

• Out of the “Total Tasks” count, all the tasks that have an Actual

Finish date

• Has an Actual Finish date entered in the IMS

• Actual Finish dates in the future (relative to the status date of the schedule), although they are considered invalid, are still counted as part of this statistic

Primavera • There is a column for Actual Finish

MS Project • There is a column for Actual Finish

Open Plan • There is a column for Actual Finish

How many tasks have been completed?

Incomplete Tasks

• Out of the “Total Tasks” count, all the tasks that do not have an

Actual Finish date

• Does not have an Actual Finish date entered in the IMS

Primavera • Actual Finish date is empty

MS Project • Actual Finish date is empty

Open Plan • Actual Finish date is empty

How many tasks are not done yet?

Baseline Count

• Out of the “Total Tasks” count, all the tasks that are baselined to finish on or before the IMS status date

Primavera • BL Project Finish date is on or before the IMS status date

MS Project • Baseline Finish date is on or before the IMS status date

Open Plan • Baseline Finish date is on or before the IMS status date

– Represents the number of tasks that should have been completed to date according to the original (baseline) plan

– Used to calculate the Missed Tasks, Metric #11

How many tasks were planned to be done by now?

BEI Baseline Count

• Out of the “Total Tasks” count, all the tasks that are baselined to finish on or before the IMS status date PLUS all the tasks that are missing baseline start or baseline finish dates

Primavera • BL Project Finish date is on or before the IMS status date or is empty

MS Project • Baseline Finish date is on or before the IMS status date or is empty

Open Plan • Baseline Finish date is on or before the IMS status date or is empty

– Represents the number of tasks that should have been completed to date according to the original (baseline) plan

– Includes tasks that are missing baseline start or finish dates

– Used to calculate the Baseline Execution Index (BEI), Metric #14

Relationship Count

• Out of the “Incomplete Tasks” count, the total number of Finish-to-

Start (FS), Finish-to-Finish (FF), Start-to-Start (SS), and Start-to- Finish (SF) relationships that exist in the schedule

Primavera • The relationship type is found in the Activity Details

MS Project • The relationship type is found in the Unique ID Predecessor or Unique ID Successor column

Open Plan • The relationship type is found in the Predecessor or Successor column

– Used to calculate the percentage of each of the four relationship types, Metric #4

How many total relationships are there?

Metric #1: Logic

• How do I identify tasks that are missing logic?

• Any task that meets all of the following criteria:

• Is considered an “Incomplete Task”

• And is missing a predecessor, a successor, or both

• What are the benefits of this metric?

• Helps identify how well or poorly the schedule is linked together

• Even if links exist, the logic still needs to be verified by the technical leads to ensure that the links make sense

• How do I calculate the percentage of tasks that are missing logic?

– [(# of Tasks missing Logic) / (# of Incomplete Tasks)] x 100 = %

– The number tasks without predecessors and/or successors should not exceed 5%

Primavera • Both the predecessor and successor fields should be filled, otherwise a task is missing logic

MS Project • Both the Unique ID Predecessors and Unique ID Successors fields should be filled, otherwise a task is missing logic

Open Plan • Both the Predecessors and Successors fields should be filled, otherwise a task is missing logic

All incomplete tasks should be linked

Dangling Activities (SS)

Build

Draft Draft Takes Longer

FS

FS

SS

SS

Design Design Takes Longer

Can Build finish before Draft and Draft before Design?

Solution – Dangling Activities

Draft

FF

Design

SSStart Here

Design Longer

Draft

FF

Design

SS

What If Predecessor Takes Longer?

What If Successor Takes Longer?

Draft

FF

Design

SS

Draft Longer

Logic Best Practice

• As a general rule to avoid “Dangling Activities”, each task should:

• Have at least one FS or SS relationship as its predecessor

• Have at least one FS or FF relationship as its successor

TASK Predecessor Successor

FS or SS FS or FF

Metric #2: Leads

• How do I identify tasks that have leads (negative lags)?

• Any task that meets all of the following criteria:

• Is considered an “Incomplete Task”

• And has a lead in its predecessor

• What are the benefits of this metric?

• The critical path and any subsequent analysis can be adversely affected by using leads.

The use of leads distorts the total float in the schedule and may cause resource conflicts.

Per the IMS DID, negative time is not demonstrable and should not be encouraged

• How do I document the findings of this metric?

– Using MS Excel, count the number of “Leads” that are found

– Leads should not be used; therefore, the goal for this metric is 0

Primavera • Visible only in individual task view – not as a column. Ask the program scheduler to extract this information for you

MS Project • The Unique ID Predecessors field, if filled, should not contain a lead (-23d, for example)

Open Plan • The Predecessors field, if filled, should not contain a lead (-23d, for example)

Leads may distort the critical path

Metric #3: Lags

• How do I identify tasks that have lags?

• Any task that meets all of the following criteria:

• Is considered an “Incomplete Task”

• And has a lag in its predecessor

• What are the benefits of this metric?

• The critical path and any subsequent analysis can be adversely affected by using lags. Per the IMS DID, lag should not be used to manipulate float/slack or to restrain the schedule

• Lags in and of themselves are not bad – rather, it is the misuse of lags that can lead to non-compliance

• How do I document the findings of this metric?

– Using MS Excel, count the number of “Lags” that are found

– The number relationships with lags should not exceed 5%

Primavera • Visible only in individual task view – not as a column. Ask the program scheduler to extract this information for you

MS Project • The Unique ID Predecessors field, if filled, should contain a minimal frequency of lags and the size of the lags should be minimal (< 5d)

Open Plan • The Predecessors field, if filled, should contain a minimal frequency of lags and the size of the lags should be minimal (< 5d)

The size and frequency of lags should be minimized

Metric #4: Relationship Types

• How do I identify the relationship types?

• Any task that meets all of the following criteria:

• Is considered an “Incomplete Task”

• And has a predecessor

• What are the benefits of this metric?

• The Finish-to-Start (FS) relationship type (“once the predecessor is finished, the successor can start”) provides a logical path through the program and should account for at least 90% of the relationship types being used. The Start-to-Finish (SF) relationship type is counter-intuitive (“the successor can’t finish until the predecessor starts” – huh?) and should only be used very rarely and with detailed justification

• How do I document the findings of this metric?

– By counting the number of Start-to-Start (SS), Finish-to-Finish (FF), and Start-to-Finish (SF) relationship types, the % of Finish-to-Start (FS) relationship types can be calculated

Primavera • Visible only in individual task view – not as a column. Ask the program scheduler to extract this information for you

MS Project • The Unique ID Predecessors field, if filled, should display the relationship type (FS, SS, FF, SF)

Open Plan • The Predecessors field, if filled, should display the relationship type (FS, SS, FF, SF – or it may be spelled out completely)

The FS relationship provides a logical path through the program

Metric #5: Hard Constraints

• How do I identify the constraint types?

• Any task that meets all of the following criteria:

• Is considered an “Incomplete Task”

• And has any type of constraint applied

• What are the benefits of this metric?

• Using hard constraints [Must-Finish-On (MFO), Must-Start-On (MSO), Start-No-Later-Than

(SNLT), & Finish-No-Later-Than (FNLT)] will prevent tasks from being moved to the right by their dependencies and, therefore, prevent the schedule from being logic-driven

• Soft constraints such as As-Soon-As-Possible (ASAP), Start-No-Earlier-Than (SNET), and Finish-No-Earlier-Than (FNET) enable the schedule to be logic-driven

• How do I document the findings of this metric?

– Divide the total number of “Hard Constraints” by the number of “Incomplete Tasks”

– The number of tasks with hard constraints should not exceed 5%

Primavera • Be sure to check both the Primary and Secondary Constraint for dates. Hard constraints in Primavera P6 are: Mandatory Start and Mandatory Finish

MS Project • Check the Constraint Type field. Hard constraints are: Must-Finish-On (MFO), Must- Start-On (MSO), Start-No-Later-Than (SNLT), & Finish-No-Later-Than (FNLT)

Open Plan • Check the Target Start Type and Target Finish Type fields. Hard constraints in Open Plan version 3.3 are: Start-Fixed-Target and Finish-Fixed-Target

Hard constraints prevent tasks from being logic-driven

Deadline Considerations

• What are “deadlines”?

• An MS Project feature that is used to indicate an important target date without requiring a hard constraint.

• How can deadlines adversely affect the schedule?

• Deadlines affect the total float calculations in the schedule.

• Improper management of deadlines and failure to notify of the deadlines’ existence in the schedule impairs critical path analysis if total float/slack values are used as the sole method for determining the critical path. As a result, complete and accurate critical path analysis and visibility into emerging schedule risks is compromised.

• What should I do about deadlines?

– Make sure deadlines are being managed properly and that the customer is aware of their existence before they perform critical path analysis.

Deadlines affect the total float/slack calculations

Primavera • Start On, Finish On, Start On or Before, and Finish On or Before behave like “deadlines”

MS Project • Check the Deadline field for any dates that are entered

Open Plan • Start Not Later Than, Finish Not Later Than , Start On Target, and Finish On Target behave like a “deadlines”

Metric #6: High Float

• How do I identify tasks with high float?

• Any task that meets all of the following criteria:

• Is considered an “Incomplete Task”

• And total float is greater than 44 working days (2 months)

• What are the benefits of this metric?

• A task with total float over 44 working days may be a result of missing predecessors and/or successors.

• If the percentage of tasks with excessive total float exceeds 5%, the network may be unstable and may not be logic-driven

• How do I document the findings of this metric?

– [(# of Tasks w/ High Float) / (# of Incomplete Tasks)] x 100 = %

– The number tasks with high float should not exceed 5%

Primavera • Total Float should be less than or equal to 44 working days

MS Project • Total Slack should be less than or equal to 44 working days

Open Plan • Finish Total Float should be less than or equal to 44 working days

High Float may indicate an unstable network

Metric #7: Negative Float

• How do I identify tasks with negative float?

• Any task that meets all of the following criteria:

• Is considered an “Incomplete Task”

• And total float is less than 0 working days

• What are the benefits of this metric?

• Helps identify tasks that are delaying completion of one or more milestones

• Tasks with negative float should have an explanation and a corrective action plan to mitigate the negative float.

• In the absence of hard constraints, there will never be negative float.

• How do I document the findings of this metric?

– Divide the total number of tasks with “Negative Float” by the number of “Incomplete Tasks”

– Ideally, there should not be any negative float in the schedule

Primavera • Total Float should be greater than 0 working days

MS Project • Total Slack should be greater than 0 working days

Open Plan • Finish Total Float should be greater than 0 working days

Tasks with negative float should have a corrective action plan

Metric #8: High Duration

• How do I identify tasks with high duration?

• Any task that meets all of the following criteria:

• Is considered an “Incomplete Task”

• And has a baseline duration greater than 44 working days (2 months)

• And has a baseline start date within the detail planning period or rolling wave

• What are the benefits of this metric?

• Helps to determine whether or not a task can be broken into two or more discrete tasks rather than one

• Helps to make tasks more manageable which provides better insight into cost and schedule

• How do I document the findings of this metric?

– Divide the total number of tasks with “High Duration” by the number of “Incomplete Tasks”

– The number tasks with high duration should not exceed 5%

Primavera • BL Project Finish should be less than or equal to 44 working days

MS Project • Baseline Duration should be less than or equal to 44 working days

Open Plan • Original Duration should be less than or equal to 44 working days

Tasks with high duration may be broken into two or more discrete tasks

Metric #9: Invalid Dates

• How do I identify tasks with invalid forecast or actual dates?

• Any task that meets all of the following criteria:

• Is considered a “Total Task”

• And has a forecast start/finish date prior to the IMS status date

• Or has an actual start/finish date beyond the IMS status date

• What are the benefits of this metric?

• A task should have forecast start and forecast finish dates that are in the future relative to the status date of the IMS (i.e. if the IMS status date is 8/1/09, the forecast date should be on or after 8/1/09)

• A task should NOT have actual start or actual finish dates that are in the future relative to the status date of the IMS (i.e. if the IMS status date is 8/1/09, the actual start or finish date should be on or before 8/1/09…not AFTER 8/1/09!)

• How do I document the findings of this metric?

– There should not be any invalid dates in the schedule

Primavera • Does not allow Early Start or Early Finish to remain in the past

MS Project • Early Start a/o Early Finish field dates should be in the future relative to the status date of the IMS

Open Plan • Does not allow Early Start or Early Finish to remain in the past

Accurate dates are important for good program management

Metric #10: Resources

• How do I identify tasks with resources (hours/dollars assigned)?

• Any task that meets all of the following criteria:

• Is considered an “Incomplete Task”

• What are the benefits of this metric?

• Provides verification that all tasks with baseline durations greater than zero have dollars or hours assigned

• Note: some contractors may NOT load their resources into the IMS. The IMS DID (DI-

MGMT-81650) does NOT require the contractor to load resources directly into the schedule.

• Read the contract and the Supplier EVM System Description and find out if resource loading of the schedule is required

• How do I document the findings of this metric?

– Divide the number of tasks without dollars/hours assigned by the number of “Incomplete Tasks”

Primavera • BL Project Total Cost field should not be empty for tasks with baseline duration greater than zero days

MS Project • Both the Baseline Work and Baseline Cost fields should not be empty for tasks with baseline duration greater than zero days

Open Plan • Budget_At_Completion field should not be empty for tasks with baseline duration greater than zero days

Resources tied to discrete tasks may be shown in the IMS or other documentation

Metric #11: Missed Tasks

• How do I identify a missed task?

• Any task that meets all of the following criteria:

• Is considered a “Total Task”

• And is supposed to be completed already (baseline finish date on or before the status date)

• And if the actual finish date or forecast finish date (early finish date) is after the baseline finish date Or

• Finish Variance (Early Finish minus Baseline Finish) is greater than zero

• What are the benefits of this metric?

• Helps identify how well or poorly the schedule is meeting the baseline plan

• How do I document the findings of this metric?

– Divide the number of missed tasks by the “Baseline Count” (which does NOT include the number of tasks missing baseline start or finish dates)

– The number of “Missed Tasks” should not exceed 5%

Primavera • Finish Variance field does not exist in Primavera. Subtract Early Finish minus Baseline Finish to get the Finish Variance (in days)

MS Project • Finish Variance field should not be greater than 0 days

Open Plan • Finish Variance field does not exist in Primavera. Subtract Early Finish minus Baseline Finish to get the Finish Variance (in days)

Measures performance compared to the baseline plan

Metric #12: Critical Path Test

• What is the objective of this test?

• To test the integrity of the overall network logic and, in particular, the critical path

• What are the benefits of this metric?

• If the project completion date (or other milestone) is not delayed in direct proportion

(assuming zero float) to the amount of intentional slip (600 days ~ 3 years) that is introduced into the schedule as part of this test, then there is broken logic somewhere in the network

• Broken logic is the result of missing predecessors and/or successors on tasks where they are needed

• How do I document the findings of this metric?

– The IMS passes the Critical Path Test if the project completion date (or other task/milestone) shows a very large negative total float number or a revised Early Finish date that is in direct proportion (assuming zero float) to the amount of intentional slip (600 days in this case) that was applied; repeat the test numerous times before “passing” the schedule

Primavera • Enter “600 d” into the Remaining Early Finish field for any incomplete discrete task

MS Project • Enter “600 d” into the Remaining Duration field for any incomplete discrete task

Open Plan • Enter “600 d” into the Computed Remaining Duration field for any incomplete discrete task

The final task in the critical path should slip one-for-one

Metric #13: Critical Path Length Index (CPLI)

• What is the objective of the CPLI?

• Measures “realism” of completing the contract by the contract’s baseline finish date

• What are the criteria of this metric?

• Must first verify that the critical path makes sense and that enough of the schedule metrics have indicated that the critical path is “believable”

• The “Critical Path Length” is always calculated from time now to the end of the program

• Default forecast finish is “end of program”

• Must be able to calculate total float

• The “Total Float” variable is the least amount of total float on the software-calculated critical path that is found using the “constraint method” (detailed instructions provided later in the training)

CPLI = Critical Path Length + Total Float Critical Path Length

• How do I document the findings of this metric?

– Target is "1.00“ with a threshold of “0.95”

– Greater than 1.00 = favorable; Less than 1.00 = unfavorable

CPLI gauges the realism of completing the contract on time

Metric #14: Baseline Execution Index (BEI)

• What is the objective of the BEI?

• Measures the number of tasks that were completed as a ratio to those tasks that should have been completed to date according to the original (baseline) plan

• What are the criteria of this metric?

• Any task that meets the following:

• Is considered a “Total Task”

• Numerator of the ratio: Any task with an actual finish date

• NOTE: Actual finish dates in the future (relative to the status date of the schedule), although they are considered invalid, are still counted in the numerator of the BEI equation

• Denominator of the ratio: Any task that has a baseline finish date on or before the status date PLUS any tasks missing baseline start or finish dates

• Tasks missing baseline dates should be included in the BEI Baseline Count until the problem is fixed

BEI = “Complete Tasks” _ “BEI Baseline Count”

• Analogous to EV dollarized Schedule Performance Index (SPI)

• How do I document the findings of this metric?

– Target efficiency ratio is "1.00“ with a threshold of “0.95”

– Greater than 1.00 = favorable; Less than 1.00 = unfavorable

BEI gauges the efficiency of contractor performance to plan