IPPS-A AMP QASP - DRAFT - 3 JAN 2024.pdf

CUI

Integrated Personnel and Pay System – Army

(IPPS-A)

Army Military Payroll

Quality Assurance Surveillance Plan

(QASP)

DRAFT

W91CRB-24-R-0003

December 20, 2023

Prepared by:

IPPS-A Product Manager (PdM) Future Capabilities Office

IPPS-A AMP QASP - DRAFT CUI

December 20, 2023 ii

THIS PAGE INTENTIONALLY LEFT BLANK 14

December 20, 2023 iii

Table of Contents 16

1.0 Introduction ......................................................................................................... 1 17

1.1 Purpose ............................................................................................................... 1 18

1.2 Intent .................................................................................................................... 2 19

1.3 Authority.............................................................................................................. 2 20

2.0 Contract Description .......................................................................................... 2 21

2.1 Background ......................................................................................................... 2 22

2.2 Scope of Work .................................................................................................... 3 23

2.3 Type of Contract ................................................................................................. 3 24

3.0 Roles and Responsibilities ................................................................................ 3 25

3.1 Government ........................................................................................................ 3 26

3.1.1 Contracting Officer ................................................................................... 3 27

3.1.2 Contracting Officer’s Representative ........................................................ 4 28

3.1.3 Product Manager and Associate PdM ...................................................... 4 29

3.1.4 Performance Monitors .............................................................................. 5 30

3.2 Contractor ........................................................................................................... 5 31

3.2.1 Contract Manager ..................................................................................... 5 32

3.2.2 Program Manager .................................................................................... 6 33

3.2.3 Quality Manager ....................................................................................... 6 34

4.0 Contract Quality Requirements ......................................................................... 6 35

4.1 Contractor’s Quality Control Program .............................................................. 6 36

4.2 Government Quality Assurance ........................................................................ 6 37

5.0 Government Surveillance .................................................................................. 7 38

5.1 Methods of Surveillance .................................................................................... 7 39

5.1.1 100% Inspection ....................................................................................... 7 40

5.1.2 Random Sampling .................................................................................... 7 41

5.1.3 Periodic Inspection ................................................................................... 7 42

5.1.4 Customer Feedback ................................................................................. 7 43

5.1.5 After Action Review (AAR) ....................................................................... 7 44

5.2 Performance Requirements Summary ............................................................. 7 45

5.3 Surveillance Schedule ..................................................................................... 15 46

5.4 Surveillance Instructions/Checklists .............................................................. 15 47

5.5 Surveillance Documentation ........................................................................... 15 48

December 20, 2023 iv

6.0 Nonconformance Reports (NCRs) (Attachment 1) ........................................ 16 49

6.1 Level 1 (Minor) .................................................................................................. 16 50

6.2 Level 2 (Major) .................................................................................................. 16 51

6.3 Level 3 (Critical) ................................................................................................ 17 52

6.4 Documenting NCRs .......................................................................................... 17 53

6.5 Tracking and Resolving NCRs ........................................................................ 18 54

6.6 Corrective Action Report (CAR). (Attachment 2) ........................................... 18 55

6.7 Performance Assessment Report (PAR). (Attachment 3) ............................. 18 56

7.0 Non-Conformance Terms................................................................................. 18 57

8.0 Data Analysis .................................................................................................... 19 58

9.0 Acceptance of Services ................................................................................... 20 59

10.0 COR Status Report ........................................................................................... 20 60

11.0 Contractor Manpower Reporting Initiative ..................................................... 20 61

12.0 Contractor Performance Assessment Reporting System (CPARS) ............. 20 62

13.0 Combating Trafficking In Persons (CTIP) ....................................................... 20 63

14.0 COR/Quality Assurance Surveillance Files .................................................... 21 64

15.0 QASP Changes ................................................................................................. 21 65

Attachment 1 - Nonconformance Report Worksheet ............................................... 22 66

Attachment 2 - Corrective Action Report (CAR) ....................................................... 23 67

Attachment 3 - Performance Assessment Report (PAR) ......................................... 24 68

List of Tables 71

Table 1. Evaluation Ratings Definitions ........................................................................... 8 72

Table 2. Performance Requirements Summary (Surveillance Matrix) ............................. 9 73

Table 3. Non-Conformance Metrics and Tolerance Ranges ......................................... 19 74

December 20, 2023 1

1.0 Introduction 76

The Integrated Personnel and Pay System – Army (IPPS-A) is a Defense Business 77 System acquisition program under the Milestone Decision Authority of the Army 78 Acquisition Executive. It is managed by the Army Program Executive Office Enterprise 79 Information Systems (PEO EIS), IPPS-A PMO, in Arlington, VA 22202. PEO EIS 80 develops, acquires, and deploys tactical and management Information Technology 81 systems and products and is a Systems Acquisition, Development and Integration 82 Center of Excellence. The IPPS-A mission is to develop and deliver a single, integrated, 83 multi-component Human Resources (HR) and Pay system that streamlines Army HR, 84 enhances the efficiency and accuracy of Army Personnel and Pay procedures, and 85 supports Soldiers and their families. IPPS-A is being deployed incrementally in a series 86 of software releases to the Total U.S. Army Force (Active Army, Army National Guard, 87 and Army Reserves) when complete. A major software release was deployed in 88 January 2023, providing Military Human Resource capabilities and bringing the Total 89 U.S. Army Force into a single authoritative system. This software release serves as the 90 production baseline and is in the Capability Support phase. 91

1.1 Purpose 92

The purpose of the Quality Assurance Surveillance Plan (QASP) is to describe the 93 systematic methods the Government will use to evaluate Contractor actions while 94 performing the requirements as laid out in the Performance Work Statement (PWS) for 95 the Army Military Pay (AMP). The QASP provides a means for evaluating whether the 96 Contractor is meeting the performance standards and acceptable quality levels 97 identified in the PWS and identification of surveillance methods used to document the 98 performance. 99

The QASP has been developed by IPPS-A, PEO EIS to provide direction to personnel 101 performing contract surveillance activities. Personnel performing surveillance of the 102 contract terms and conditions and PWS requirements will periodically review the QASP 103 throughout the life of the contract. The QASP establishes procedures on how this 104 assessment/inspection process will be conducted. It provides details for a 105 continuous oversight process as follows: 106

What will be monitored 107 How monitoring will take place 108 Who will be conduct the monitoring 109 How monitoring efforts and results will be documented 110

The Contractor is responsible for implementing and delivering performance that meets 111 contract standards using its Quality Control Plan. The QASP provides the structure for 112 the Government's surveillance of the Contractor's performance to assure that it meets 113 contract standards. It is the Government's responsibility to be objective, fair and 114 consistent in evaluating Contractor performance. 115

December 20, 2023 2

The QASP is not part of the contract, nor is it intended to duplicate the Contractor's 117 quality control plan. This QASP is a living document. Flexibility in the QASP is required 118 to allow for an increase or decrease in the level of surveillance necessary based on 119 Contractor performance. The Government will provide a copy of the QASP to the 120 Contractor to facilitate open communication. In addition, the QASP should recognize 121 that unforeseen or uncontrollable circumstances might occur that are outside the control 122 of the Contractor. As a result, the QASP should ensure early identification and 123 resolution of performance issues to minimize impact on mission performance. 124

1.2 Intent 126

The intent of this QASP is to: 1) ensure Contractor performance meets or exceeds 127 contract terms, conditions, and specifications; 2) hold the Contractor accountable for 128 quality control; and 3) encourage the Contractor to take appropriate steps to control and 129 improve quality. Accordingly, the Government will perform surveillance on the contract 130 in accordance with this QASP; and reserves the right to monitor the contract in any 131 manner necessary, at any times necessary, and at all places necessary to ensure that 132 the rendered services conform to contract requirements. The Government also reserves 133 the right to perform quality assurance at the Subcontractor level and perform quality 134 assurance at the Contractor's place of business, if applicable. Resolution of non-135 conforming services discovered at the Subcontractor level will be addressed with the 136 prime Contractor. 137

1.3 Authority 138

Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition 139 Regulation, Inspection of Services clauses, which provides for inspection, acceptance 140 and documentation of the service called for in the contract or order. This acceptance is 141 to be executed by the Contracting Officer or a duly authorized representative. 142

2.0 Contract Description 143

2.1 Background 144

This is a non-personal services contract to obtain Contractor services to develop an 145 Army Military Payroll (AMP) solution using the Oracle PeopleSoft Global Payroll 146 application. The Army launched the Integrated Personnel and Pay System – Army 147 (IPPS-A) Increment II (Inc II) production baseline for Human Resources (HR) 148 processing in December 2023 utilizing PeopleSoft Human Capital Management (HCM). 149 The new AMP capability will be integrated with the IPPS-A production baseline. 150

Development will consider (1) Agile execution combined with Development, Security, 151 and Operations (DevSecOps) for all activities / tasks to allow for Continuous Integration 152 / Continuous Delivery (CI/CD) and compliance with the Continuous Risk Management 153 Framework (cRMF), (2) task automation to reduce / eliminate manual effort, and (3) 154 innovation and creativity in problem-solving. 155

December 20, 2023 3

For this contract, Agile execution is defined as a group of software methods (e.g., 156 Scrum, Kanban, SAFe®) based on iterative and incremental development and 157 deployment of software capabilities for end users, where requirements and solutions 158 evolve through collaboration between cross-functional teams. These teams include the 159 Government and Contractor communities working across parallel environments where 160 capabilities are implemented, and production system capability support is ongoing. 161

Contractors will apply and perform Agile execution in varying degrees to all aspects of 162 the program life cycle across and within key activities. These activities include 163 requirements analysis, elicitation, elaboration, and decomposition; architecture; design; 164 development, implementation; test; integration; training material development; and 165 deployment, in addition to support functions such as program management (planning, 166 reporting), configuration management, change control, release management, quality 167 assurance, and cybersecurity. 168

2.2 Scope of Work 169

The Army requires the following services, and the Contractor shall deliver an AMP 170 solution as follows: (1) design, configure, build, and test, (2) integrate and test the AMP 171 solution with the IPPS-A production baseline, (3) develop AMP training material, (4) 172 support AMP release and deployment activities, and (5) transition AMP functionality to 173 capability support. 174

These services will be acquired across the following tasks: 175

Task 1: Program Management 176 Task 2: Solution Configuration, Development, and Test 177 Task 3: Solution Integration 178 Task 4: Release and Deployment Support 179 Task 5: Government Software Reuse Integration 180 Task 6: Training Material Development 181 Task 7: Capability Support Transition 182 Task 8: Training Delivery [OPTIONAL] 183

2.3 Type of Contract 184

This is a Hybrid contract including Firm Fixed Price (FFP), Cost Plus Fixed Fee (CPFF) 185 Level of Effort (LOE), CPFF Completion, and Cost Reimbursable (no fee) line items for 186 Travel and Other Direct Costs (ODCs). 187

3.0 Roles and Responsibilities 188

3.1 Government 189

3.1.1 Contracting Officer 190

The Contracting Officer (KO) shall ensure performance of all necessary actions for 191 effective contracting and compliance with the contract terms and shall safeguard the 192 interests of the United States in the contractual relationship. The KO shall also ensure 193

December 20, 2023 4 the Contractor receives impartial, fair, and equitable treatment under this contract. The 194 KO shall determine and document the final assessment of the Contractor's performance 195 in the Government’s past performance tracking system. 196

Assigned CO: Kristin M. Height, Contracting Officer 197 Organization or Agency: U.S. Army Contracting Command – Aberdeen Proving Ground 198 Telephone: (443) 861-4613 199 Email: kristin.m.height.civ@army.mil 200

3.1.2 Contracting Officer’s Representative 201

The Contracting Officer’s Representative (COR) will be identified by separate letter. The 202 COR monitors all technical aspects of the contract and assists in contract 203 administration. The COR is authorized to perform the following functions: assure that 204 the Contractor performs the technical requirements of the contract: perform inspections 205 necessary in connection with contract performance: maintain written and oral 206 communications with the Contractor concerning technical aspects of the contract: issue 207 written interpretations of technical requirements, including Government drawings, 208 designs, specifications: monitor Contractor's performance and notifies both the 209 Contracting Officer and Contractor of any deficiencies; coordinate availability of 210 Government furnished property, and provide site entry of Contractor personnel. A letter 211 of designation issued to the COR, a copy of which is sent to the Contractor, states the 212 responsibilities and limitations of the COR, especially with regard to changes in cost or 213 price, estimates or changes in delivery dates. The COR is not authorized to change any 214 of the terms and conditions of the resulting order. 215

Assigned COR: Dawn Bare, Contracting Officer Representative 216 Organization or Agency: USARMY HQDA IPPS-A 217 Telephone: (703) 545-2743 218 Email: dawn.m.bare2.civ@army.mil 219

3.1.3 Product Manager and Associate PdM 220

The IPPS-A Product Manager (PdM) and AMP Associate PdM (APM) provide primary 221 program oversight. While the PdM and APM may serve as a direct conduit to provide 222 Government guidance and feedback to the Contractor on project matters, the PdM and 223 APM are not empowered to make any contractual commitments or any contract 224 changes on behalf of the Government. The PdM and APM will recommend a COR to 225 the KO, ensure the COR is trained before performing any delegated duties, and will 226 support COR performance assessment activities. 227

Product Manager: LTC Ryan Martin 228 Organization or Agency: USARMY HQDA IPPS-A 229 Telephone: (703) 545-3291 230 Email: ryan.p.martin7.mil@army.mil 231 Associate Product Manager: MAJ Lauri Blair 232 Organization or Agency: USARMY HQDA IPPS-A 233 Telephone: (832) 353-9695 234 Email: lauri.r.blair.mil@army.mil 235

December 20, 2023 5

3.1.4 Performance Monitors 236

3.1.4.1 Functional Monitor 237

The AMP Functional Monitor will provide detailed functional oversight of the 238 Contractor’s performance regarding business value and alignment with the AMP 239 functional requirements and report his or her findings to the COR in a timely, complete, 240 and impartial fashion to support the COR’s administration activities. Any changes that 241 the Contractor deems may affect contract, price, terms, or conditions shall be referred to 242 the Contracting Officer for action. The Functional Monitor is not empowered to provide 243 any guidance to the Contractor or make any contractual commitments or any contract 244 changes on behalf of the Government. 245

Functional Monitor: LTC Erica Miller 246 Organization or Agency: USARMY HQDA ASA MRA 247 Telephone: (703) 614-8208 248 Email: erica.l.miller.mil@army.mil 249

3.1.4.2 Technical Monitors 250

AMP Technical Monitors will provide detailed technical oversight of the Contractor’s 251 performance and report their findings to the COR in a timely, complete, and impartial 252 fashion to support the COR’s administration activities. Any changes that the Contractor 253 deems may affect contract, price, terms, or conditions shall be referred to the 254 Contracting Officer for action. Technical monitors may include performance monitors, 255 inspectors, technical experts, or others who provide information that helps the COR 256 monitor Contractor performance. The Technical Monitors are not empowered to provide 257 any guidance to the Contractor or make any contractual commitments or any contract 258 changes on behalf of the Government. 259

Lead Technical Monitor: Katherine Seifert 260 Organization or Agency: USARMY HQDA IPPS-A 261 Telephone: (571) 314-4046 262 Email: katherine.e.seifert.civ@army.mil 263

3.2 Contractor 264

3.2.1 Contract Manager 265

The Contract Manger manages the full range of acquisition activities. This individual will 266 be the primary point of contact for the KO and any designated CORs. Additionally, the 267 Contract Manger will 1) perform work related to Department of Defense (DoD) Contracts 268 Management, such as FAR, Defense Federal Acquisition Regulation Supplement 269 (DFARS), DoD Instructions (DoDI) 5000 series regulations and Clinger-Cohen Act 270 compliance; 2) track deadlines, deliverables, and other obligations; 3) plan, direct, and 271 manage projects/operations; 4) track and maintain a record of modifications; 4) record 272 and solve contract problems/issues; 5) report on contract deliverables and performance, 273 and 6) ensure product quality. In addition, the Contract Manager will be knowable of 274 different contract types; and will coordinate and interface with the Government 275 contracting office on contract-related action items and management activities. 276

December 20, 2023 6

Contract Manager: TBD 277 Telephone: 278 Email: 279

3.2.2 Program Manager 280

The Contractor will assign an individual to serve as a Program Manager (PM) for this 281 effort. This individual will be the single point of contact for the Government Task Monitor 282 (GTM) and COR. The PM will be responsible for the performance of the work. The 283 name of this person and an alternate, who will act for the Contractor when the PM is 284 absent, will be designated in writing to the Contracting Officer. The Contractor PM and 285 alternate will have full authority to act for the Contractor on all matters relating to daily 286 operation of this contract. 287

Program Manager: TBD 288 Telephone: 289 Email: 290

3.2.3 Quality Manager 291

The Quality Manager will maintain an effective quality control program to ensure 292 services are performed as agreed to in the contract. This will include the following: 293

Updating and implementing procedures to identify, prevent, and ensure the non-294 recurrence of defective services, to include products and software 295

Using programmatic and technical metrics as quality measures to assess 296 performance, progress, speed, cybersecurity, and quality of the product team, 297 leveraging automated tools to the maximum extent practicable 298

Quality Manager: TBD 299 Telephone: 300 Email: 301

4.0 Contract Quality Requirements 302

4.1 Contractor’s Quality Control Program 303

The Contractor’s quality control program (IAW FAR Part 46 and applicable 52.246 304 Clause(s)) is the means by which the Contractor assures that work complies with the 305 requirement of the contract. The Contractor shall ensure services are performed in 306 accordance with the PWS. The Contractor shall identify, prevent, and ensure non-307 recurrence of defective services. 308

4.2 Government Quality Assurance 309

The Government shall evaluate the Contractor’s performance under this contract in 310 accordance with this QASP. This plan is primarily focused on what the Government 311 must do to ensure that the Contractor has performed in accordance with the 312 performance standards. It defines how the performance standards will be applied, the 313 frequency of surveillance and minimum acceptable defect rate(s). 314

December 20, 2023 7

5.0 Government Surveillance 315

5.1 Methods of Surveillance 316

The following Methods of Surveillance will be utilized in monitoring the Contractors’ 317 performance: 318

5.1.1 100% Inspection 319

The Government will inspect 100% of each deliverable or service performed. This 320 surveillance method is appropriate where 100% inspection is required and is practical. 321

5.1.2 Random Sampling 322

The Government will randomly sample less than 100% of a designated deliverable or 323 performed service on a random basis. This surveillance method is appropriate for a 324 deliverable or performed service that occurs infrequently and where 100% inspection is 325 neither required nor practical. 326

5.1.3 Periodic Inspection 327

The Government will inspect less than 100% of a designated deliverable or performed 328 service, on a periodic basis. This surveillance method is appropriate for a deliverable or 329 performed service that occurs infrequently and where 100% inspection is neither 330 required nor practical. 331

5.1.4 Customer Feedback 332

The following procedures will be followed when a complaint is received regarding the 333 Contractor/Contractor Employees. 334

The COR will investigate and determine the validity of the complaint 335

If the complaint is determined to be invalid, the COR will inform the person who 336 submitted the complaint of the reason(s) as soon as practical 337

For validated complaints that are true contractual non-conformances, the 338 procedures in paragraph 6 below will be followed 339

5.1.5 After Action Review (AAR) 340

The Government will perform random AAR’s at the Planning Interval (PI) and/or sprint 341 level up to 100% of PI’s exercised. This surveillance method is appropriate for a 342 deliverable or performed service which will capture best practices, bottlenecks, and 343 future PI/sprint development processes for implementation. 344

5.2 Performance Requirements Summary 345

In evaluating the quality of the Contractor’s performance, the following performance 346 ratings under Table 1 may be used IAW FAR 42.1503, Table 42-1. 347

December 20, 2023 8

Table 1. Evaluation Ratings Definitions 348

Rating Definition Note

Exceptional – Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with few minor problems for which corrective actions taken by the Contractor were highly effective.

To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an Exceptional rating. Also, there should not have been any significant weaknesses identified.

Very Good – Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the Contractor were effective.

To justify a Very Good rating, identify a significant event and state how it was a benefit to the Government. There should not be any significant weaknesses identified.

Satisfactory – Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the Contractor appear or were satisfactory.

To justify a Satisfactory rating, there should have been only minor problems, or major problems the Contractor recovered from without impact to the contract/order.

There should not have been significant weaknesses identified. A fundamental principle of assigning ratings is that Contractors will not be evaluated with a rating lower than Satisfactory solely for not performing beyond the requirements of the contract/order.

Marginal – Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a serious problem for which the Contractor has not yet identified corrective actions. The Contractor proposed actions appear only marginally effective or were not fully implemented.

To justify Marginal performance, identify a significant event in each category that the Contractor had trouble overcoming and state how it impacted the Government. A Marginal rating should be supported by referencing the management tool that notified the Contractor of the contractual deficiency (e.g., management, quality, safety, or environmental deficiency report or letter).

Unsatisfactory – Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which Contractor corrective actions appear or were ineffective.

To justify an Unsatisfactory rating, identify multiple significant events in each category that the Contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating. An Unsatisfactory rating should be supported by referencing the management tools used to notify the Contractor of the contractual deficiencies (e.g., management, quality, safety, or environmental deficiency reports, or letters).

The Performance Requirements Summary or Surveillance Matrix (Table 2) is the list of 349 performance objectives and standards that must be performed by the Contractor. This 350 matrix details the method of surveillance the COR will use to validate and inspect these 351 performance elements. Inspection of each element will be documented in the COR file. 352 The Government will monitor quality measures monthly using agile development tools to 353 the extent available and will formally review quality measures at the end of each PI 354 utilizing agile development tools reporting to the extent available. The Government will 355 consider updating metrics collaboratively with the Contractor post-award and iteratively 356 throughout the period of performance. 357

Performance objectives define the desired outcomes. Performance Standards define 358 the level of service required under the contract to successfully meet the performance 359 objective. The inspection methodology defines how, when, and what will be assessed in 360 measuring performance. The Government performs surveillance, using this QASP, to 361 determine the quality of the Contractor's performance as it relates to the performance 362 element standards. The Program Requirements Summary will be used to form the 363 foundation of the COR's inspection checklist. 364

December 20, 2023 9

Table 2. Performance Requirements Summary (Surveillance Matrix) 365

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

Program Management

PWS Section(s):

5.1 - 5.8

Deliverables:

All

Program Planning and Reporting

Quality Control and Management

Risk Management

Schedule Management

Personnel Management

Cost and Software Data Reporting

Contract Funds Management

License Management

Property Management

Knowledge Management

Periodic and ad hoc Meetings and Reporting

RA: Management

Program Management activities span all contract tasks. This includes the management and oversight of all activities performed by Contractor personnel, including Subcontractors, to satisfy the requirements identified in the PWS.

Accurate, complete. and timely support of tasks and functions as directed by the Government Implementation of the schedule Milestones met on time Requirements met within proposed cost Metric reports delivered on time in correct format/content

Meetings and Reporting provides the information needed

Task activities defined in PWS successfully completed

Very Good: meets the following at a minimum of 95%:

Meets milestones as scheduled Meets contract cost baseline Personnel required to perform tasks are qualified and present

Exceptional: meets the following at a minimum of 97%:

Meets milestones as scheduled Meets contract cost baseline Personnel required to perform tasks are qualified and present

100% compliant with all procedures and applicable regulations

90% of Contract Data Requirements List (CDRLs) delivered on time and in compliance with the DD FORM 1423-1

Satisfactory: meets the following at a minimum of 93%:

Meets milestones as scheduled

Meets contract cost baseline

Personnel required to perform tasks are qualified and present

100% Inspection CDRLs Staffing Levels and Quality of Performance

Outcomes and performance of regularly scheduled and ad hoc meetings

Contract Performance

PWS Section(s):

5.1

Deliverables:

A003 A009

Roadmap / Schedule

RA: Schedule

Measures how efficiently the project is progressing compared to the planned timeline

Roadmap / Schedule Performance = Actual number of Scheduled PI Activities and Tasks / Planned number of Scheduled PI Activities and Tasks

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Cost

RA: Cost Control

Measures the cost efficiency of the project by comparing planned with actual costs

Cost Performance = Actual Cost / Planned Cost

Very Good: 95% to <100%

Exceptional: <95%

Satisfactory: 100% 100% Inspection

December 20, 2023 10

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

Small Business Utilization

RA: Small Business Subcontracting

Measures the actual achievements relative to Contractor proposed Small Business Participation Commitment Document by business category

Small Business Utilization = Cumulative percentage total by adding the Percent of total contract value for each business category

Very Good: 26%

Exceptional: 27%

Satisfactory: 25% 100% Inspection

Throughput

PWS Section(s):

5.2 - 5.4

Deliverables:

A003 B006 B012 B014

Story Point Completion Rate (Predictability)

RA: Quality

Measures the rate at which story points are being completed within a sprint and PI

Story Point Completion Rate = the number of story points completed / number of story points planned per sprint/PI

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Story Point Velocity

Measures the number of story points completed versus the Average Velocity per sprint or PI

Average Velocity = total number of story points completed across all sprints or PIs / total number of sprints or PIs

Story Point Velocity = number of Story Points completed in a sprint or PI / Average Velocity per sprint or PI

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Story Completion Rate stories completed in a sprint or PI

Story Completion Rate = number of stories completed / number of stories committed to (agreed upon in the plan) within a sprint or PI

This metric Threshold and Objective percentages may be adjusted if the number of total stories across Agile Teams is less than 20.

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Release Burnup

Measures the amount of work completed for a release (Note: work should increase as sprints are completed until release)

Release Burnup = sum of work completed in sprints associated with a specific release / planned work in the specific release

Very Good: 95% at the end of a specific release

Exceptional: 97% at the end of a specific release

Satisfactory: 93% at the end of a specific release

100% Inspection

December 20, 2023 11

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

Recidivism stories returned to the team

Recidivism = number of stories returned in a PI / number of stories completed in a PI

Very Good: 6%

Exceptional: 4%

Satisfactory: 8% 100% Inspection

Software Quality

PWS Section(s):

5.2 - 5.4

Deliverables:

A003 B014 D001

First-time Pass stories that pass the first time in a sprint or PI

First-time Pass Rate = number of stories passed (completed and accepted) in accordance with Definition of Done in a sprint or PI / number of completed stories in a sprint or PI

Very Good: 95%

Exceptional: 97%

Defect Density defects per PI

Defect Density = number of defective stories in a PI / number of stories completed in a PI

Very Good: 6%

Exceptional: 4%

Satisfactory: 8% 100% Inspection

Escaped Defect Density escaped defects per release to the integration environment

Escaped Defect Density = number of escaped defective stories / total number of stories completed in the release

Very Good: 6%

Exceptional: 4%

Satisfactory: 8% 100% Inspection

Recurring Defects

Measures the percentage of recurring defects at the end of each PI

Total Recurring Defects Percentage = Total Number of Recurring Defects Identified over all PIs / Total Number of Defects Identified over all PIs

Very Good: TBD

Exceptional: TBD

Satisfactory: TBD 100% Inspection

Defect Closure of total defects closed at the end of each PI

Defect Closure Rate = Defects Closed (Identified in a PI or Carried from a previous PI) / Total Defects per PI (Identified in a PI plus Carried from a previous PI)

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Number of Blockers (anything that impedes completion of a story, activity, or work item)

Measures the number of events that prohibit completion of a story, activity, or work item

Number of Blockers = Number of blockers currently open that prohibit

Very Good: 6 blockers

Exceptional: 4 blockers

Satisfactory: 8 blockers

100% Inspection

December 20, 2023 12

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of the completion of a story, activity or work item in a sprint or PI

Integration (Interoperability) Testing of stories that pass integration testing within the integration environment

Integration (Interoperability) Testing = number of stories that pass integration testing / number of stories included in integration testing

Very Good: 97%

Exceptional: 99%

Satisfactory: 95% 100% Inspection

Delivered Features (or Capabilities)

Measures the delivery progress of each Feature relative to the planned completion rate of each Feature in a PI

Delivered Features (or Capabilities) = % of each Feature (or Capability) completed in a PI / % of each Feature planned in that PI

Very Good: 95%

Exceptional: 97%

Value Delivered

PWS Section(s):

5.2, 5.8

Deliverables:

A003

Delivered Value (Business Value Delivered)

Measures Business Value Delivered in a Sprint / PI

Business Value Delivered = total business value points completed / total business value points planned

Very Good: 93%

Exceptional: 97%

Satisfactory: 90% 100% Inspection

Cybersecurity

PWS Section(s):

5.2, 5.3

Deliverables:

A003 B009 C001

RMF Actions

The Contractor shall follow DoD and Army RMF policies and regulations that govern the Department of Defense (DOD) RMF process (risk identification; risk measurement and assessment; risk mitigation; risk reporting and monitoring; and risk governance) Conduct risk reduction activity by scanning delivered code to identify cybersecurity vulnerabilities and to modify code to reduce / eliminate vulnerabilities.

For vulnerabilities that cannot be eliminated, assist with RMF actions to identify risk, risk measurement and assessment, risk mitigation, risk reporting and monitoring (i.e., POAMs and eMASS documentation)

Very Good: 96%

Exceptional: 100%

December 20, 2023 13

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of

RMF Actions = Sum of total number of eliminated vulnerabilities and total number of vulnerabilities with POAMs / total number of identified vulnerabilities

Training Material Development

PWS Section(s):

5.6, 5.8

Deliverables:

A003 F001 F002

Training Material Effectiveness

Measures the training materials’ effectiveness to convey the knowledge to users

Accurate and complete Training Materials

Training proficiency levels identified in the agreed to Training Plan are met

Very Good: All training participants achieve a score of 80% or better and an average score of 82%

Exceptional: All training participants achieve a score of 80% or better and an average score of 85%

Satisfactory: All training participants achieve a score of 80% or better and an average score of 80%

100% Inspection of user training scores

Training

PWS Section(s):

5.8

Deliverable:

A003

Training experience satisfaction

Measures training satisfaction using post-training survey. It is used to gauge how satisfied learners are with the training they received.

Training experience satisfaction = sum of all questionnaire scores / number of questionnaire scores

Very Good: 87%

Exceptional: 90%

Satisfactory: 83% 100% Inspection

Transition

PWS Section(s):

5.1, 5.7

Deliverables:

A003 A008 G001

Follow the Transition Plan accepted by the Government and keep the Government fully informed of status throughout the Transition period.

Transition

Planning and Management

Transition Accountability

Transition Activity

Collaboration Asset Transfer

Transfer responsibility for project documentation, resources, assets, and performance to the designated party. Ensure the implementation and readiness of capabilities necessary for all aspects of performance without disruption in schedule, increased costs, degradation to performance, need for increased Government oversight, or likelihood of unsuccessful

Accurate, complete. and timely support of tasks and functions as directed by the Government Products delivered on time in correct format/content Meetings and Reporting provides the information needed

Task activities defined in PWS successfully completed

Very Good: meets the following at a minimum:

100% compliant with all procedures and applicable regulations

100% of CDRLs delivered on time and in compliance with the DD

FORM 1423-1.

95% Personnel required to perform tasks are qualified and present

Exceptional: meets the following at a minimum:

100% compliant with all procedures and applicable regulations

100% of CDRLs delivered on time and in compliance with the DD

FORM 1423-1.

97% Personnel required to perform tasks are qualified and present

Satisfactory: meets the following at a minimum:

100% compliant with all procedures and applicable regulations

90% of CDRLs delivered on time and in compliance with the DD

FORM 1423-1.

93% Personnel required to perform tasks are qualified and present

100% Inspection:

CDRLs Outcomes and performance of regularly scheduled and ad hoc meetings

December 20, 2023 14

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

AMP

Performance Standards

PWS Section(s):

5.3

Deliverable:

A003 B009 B010 C001

Accuracy of Soldiers that received accurate pay, error free, across processed payroll transactions, such as Soldier data payroll calculation tax calculations rations allotments garnishments Thrift Savings Plan Government liabilities Others

Accuracy = number of Soldiers that receive accurate pay (error free) / number of Soldiers receiving pay

Very Good: Not Applicable

Exceptional: 100%

Satisfactory: 99% 100% Inspection

Timeliness

Measures the time it takes for the system to process payroll transactions for 1M+ payees without impacting day to day operations within the IPPS- A baseline

Very Good: 6 hours

Exceptional: 4 hours

Satisfactory: 8 hours 100% Inspection

Compliance of payroll transactions that comply with LPRDs:

laws policies regulations directives

Compliance = Number of complied with LPRDs / Total number of LPRDs

Very Good: 97%

Exceptional: 99%

Satisfactory: 95% 100% Inspection

Solider Satisfaction

Measures Soldier satisfaction with the payroll process; can be assessed through surveys, feedback forms, or focus groups.

A high level of satisfaction indicates that the system meets the needs of Soldiers and their families.

Soldier Satisfaction = (total survey point score / total achievable points)

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

*Note, the Government notifies the Contractor that it intends to adopt automated testing and will establish metric(s) for automated 366 testing at a later date under the terms of QASP paragraph 15.0. 367

In addition, the Government will inspect user stories (hereafter stories) at the end of 368 each Sprint to evaluate the “Definition of Done” achieved during the Sprint. The target 369 “Definition of Done” for each Sprint will be defined at the beginning of the Sprint up to 370 and including integration and interoperability criteria. The Government expects that 371 stories for interoperability between applications and interfaces will be explicitly identified 372 as part of the development process. 373

December 20, 2023 15

A CPARS rating area (RA) is assigned to each relevant metric. The rating areas are 374 Quality, Management, Schedule, Cost Control, Small Business Subcontracting, 375 regulatory compliance. Each rating area rating will be provided to the COR as input for 376 monthly and annual reporting. 377

The COR, along with the IPPS-A PdM, AMP APM, and designated Government 378 Functional and Technical Monitors, will inspect all tasks to ensure Contractor 379 compliance with the requirements at the conclusion of each Sprint or PI. Delivery will 380 occur by a data pull within the AMP agile development tools repository. Inspection 381 results will be recorded and reported. The COR may find the delivery satisfactory even 382 though further work is required, provided that the specific requirements of the sprint are 383 met. 384

At the conclusion of each sprint, the COR, along with the IPPS-A PdM, AMP APM, and 385 designated Government Functional and Technical Monitors, will review the completed 386 stories and related functionality. Incomplete or inadequate code and stories will be 387 noted in a mutually agreed-upon issue tracker, such as Trello or GitHub Issues, and 388 links to each issue shared with the COR. The Contractor may respond in that tracker as 389 appropriate, addressing the accuracy and validity of the defect as well as any planned 390 corrective action (if not already noted). The Contractor team will discuss and document 391 actions to prevent recurrence in their sprint retrospectives. 392

At the conclusion of the period of performance, a similar procedure will be followed to 393 document discrepancies and to assess overall performance. 394

5.3 Surveillance Schedule 395

A Surveillance Schedule will be created for each month of Contractor performance and 396 filed in the Procurement Integrated Enterprise Environment (PIEE), Surveillance and 397 Performance Monitoring (SPM), Surveillance Documents Section of the Contract File 398 using the Add Smart Form function. The schedule is “Controlled Unclassified 399 Information (CUI)” and is not releasable to anyone other than those authorized by the 400 Government. 401

5.4 Surveillance Instructions/Checklists 402

Surveillance Checklists will be completed when surveillances are conducted and filed in 403 the PIEE, SPM, Surveillance Documents Section of the Contract File using the Add 404 Smart Form function. Supplementary data, such as end of sprint reports directly 405 extracted from the development tools that provide a snapshot of real-time progress will 406 be captured when possible. The objective is to take advantage of any tools and 407 streamlined processes available that capture results suitably to obtain the most 408 accurate results in an administrative-efficient manner. 409

5.5 Surveillance Documentation 410

No later than the 15th of each month surveillance documentation for the preceding 411 month will be filed in the PIEE, SPM, Surveillance Documents Section. Surveillance 412 documentation includes a COR Status Report, the Surveillance Schedule, all completed 413 Surveillance Checklist(s) for the period the COR Status Report covers, 414

December 20, 2023 16

Nonconformance Reports, Performance Assessment Report, Correction Action Report, 415 if any, and Contractor Deliverables, if any. 416

6.0 Nonconformance Reports (NCRs) (Attachment 1) 417

Nonconformance occurs when a Contractor fails to meet Contract or Task Order terms, 418 conditions and/or specifications or the services are defective in some manner. All 419 instances of nonconformance will be documented via a Non-conformance Report 420 Worksheet, Attachment 1 and the Contracting Officer will notify the Contractor of the 421 nonconforming performance via Letter. Resolution of nonconformance discovered at the 422 Subcontractor level will be addressed with the prime Contractor. Nonconformance is 423 classified as either: Level 1 (Minor); Level 2 (Major); or Level 3 (Critical). NCRs may 424 contain multiple nonconformances, each characterized by a level and type. The level of 425 the NCR should normally correspond to the highest level of nonconformance listed in 426 the NCR. For example, if an NCR had one Level 3, one Level 2, and three Level 1 non-427 conformances, the NCR level would be Level 3. The exception might be when the 428 aggregate of multiple nonconformances considered in whole associate to a more 429 serious type of nonconformance defined in FAR 46.101 (e.g., if an NCR had three Level 430 2 nonconformances but the definition of critical nonconformance better describes the 431 impact/effect when considered in whole, the NCR level would be 3). 432

6.1 Level 1 (Minor) 433

Level 1 nonconformances are those that are ‘minor’ in nature such that they can be 434 promptly corrected by the Contractor, present no need for root cause analysis or 435 determination if other contract requirements are affected, present no need for further 436 preventive action, and do not materially reduce the usability of the supplies, services, 437 construction, or deliverables. While the Contractor must correct the nonconformance, 438 further actions and Government follow-up are not required, and no written response 439 from the Contractor is needed. 440

6.2 Level 2 (Major) 441

Level 2 nonconformance(s) are those that are ‘major’ in nature such that they are not 442 promptly correctable, and/or warrant root cause analysis and preventive action, and/or 443 need action by the Contractor to determine if other contract requirements are affected. 444 These nonconformances are serious enough to warrant not only correction of the 445 nonconformance, but also a written response from the Contractor in the form of a 446 Corrective Action Plan (CAP), and Government follow-up. The qualifying distinction 447 between a Level 1 and Level 2 nonconformance is the need for a CAP and Government 448 follow-up. An NCR identifying a Level 2 nonconformance may also be issued after 449 repetitive Level 1 instances of the same or similar nonconformance demonstrate the 450 Contractor is failing to institute effective corrective action and/or failing to implement 451 effective quality and/or internal control(s).