Attachment 5 - IPPS-A AMP QASP - Final - 12 JAN 2024.pdf

CUI

Integrated Personnel and Pay System – Army

(IPPS-A)

Army Military Payroll

Quality Assurance Surveillance Plan

(QASP)

W91CRB-24-R-0003

January 12, 2024

Prepared by:

IPPS-A Product Manager (PdM) Future Capabilities Office

IPPS-A AMP QASP CUI

January 12, 2024 ii

THIS PAGE INTENTIONALLY LEFT BLANK 14

January 12, 2024 iii

Table of Contents 16

1.0 Introduction ......................................................................................................... 1 17

1.1 Purpose ............................................................................................................... 1 18

1.2 Intent .................................................................................................................... 2 19

1.3 Authority.............................................................................................................. 2 20

2.0 Contract Description .......................................................................................... 2 21

2.1 Background ......................................................................................................... 2 22

2.2 Scope of Work .................................................................................................... 3 23

2.3 Type of Contract ................................................................................................. 3 24

3.0 Roles and Responsibilities ................................................................................ 3 25

3.1 Government ........................................................................................................ 3 26

3.1.1 Contracting Officer ................................................................................... 3 27

3.1.2 Contracting Officer’s Representative ........................................................ 4 28

3.1.3 Product Manager and Associate PdM ...................................................... 4 29

3.1.4 Performance Monitors .............................................................................. 5 30

3.2 Contractor ........................................................................................................... 5 31

3.2.1 Contract Manager ..................................................................................... 5 32

3.2.2 Program Manager .................................................................................... 6 33

3.2.3 Quality Manager ....................................................................................... 6 34

4.0 Contract Quality Requirements ......................................................................... 6 35

4.1 Contractor’s Quality Control Program .............................................................. 6 36

4.2 Government Quality Assurance ........................................................................ 6 37

5.0 Government Surveillance .................................................................................. 7 38

5.1 Methods of Surveillance .................................................................................... 7 39

5.1.1 100% Inspection ....................................................................................... 7 40

5.1.2 Random Sampling .................................................................................... 7 41

5.1.3 Periodic Inspection ................................................................................... 7 42

5.1.4 Customer Feedback ................................................................................. 7 43

5.1.5 After Action Review (AAR) ....................................................................... 7 44

5.2 Performance Requirements Summary ............................................................. 7 45

5.3 Surveillance Schedule ..................................................................................... 15 46

5.4 Surveillance Instructions/Checklists .............................................................. 15 47

5.5 Surveillance Documentation ........................................................................... 16 48

January 12, 2024 iv

6.0 Nonconformance Reports (NCRs) (Attachment 1) ........................................ 16 49

6.1 Level 1 (Minor) .................................................................................................. 16 50

6.2 Level 2 (Major) .................................................................................................. 16 51

6.3 Level 3 (Critical) ................................................................................................ 17 52

6.4 Documenting NCRs .......................................................................................... 17 53

6.5 Tracking and Resolving NCRs ........................................................................ 18 54

6.6 Corrective Action Report (CAR). (Attachment 2) ........................................... 18 55

6.7 Performance Assessment Report (PAR). (Attachment 3) ............................. 18 56

7.0 Non-Conformance Terms................................................................................. 19 57

8.0 Data Analysis .................................................................................................... 19 58

9.0 Acceptance of Services ................................................................................... 20 59

10.0 COR Status Report ........................................................................................... 20 60

11.0 Contractor Manpower Reporting Initiative ..................................................... 20 61

12.0 Contractor Performance Assessment Reporting System (CPARS) ............. 20 62

13.0 Combating Trafficking In Persons (CTIP) ....................................................... 21 63

14.0 COR/Quality Assurance Surveillance Files .................................................... 21 64

15.0 QASP Changes ................................................................................................. 21 65

Attachment 1 - Nonconformance Report Worksheet ............................................... 22 66

Attachment 2 - Corrective Action Report (CAR) ....................................................... 23 67

Attachment 3 - Performance Assessment Report (PAR) ......................................... 24 68

List of Tables 71

Table 1. Evaluation Ratings Definitions ........................................................................... 8 72

Table 2. Performance Requirements Summary (Surveillance Matrix) ............................. 9 73

Table 3. Non-Conformance Metrics and Tolerance Ranges ......................................... 19 74

January 12, 2024 1

1.0 Introduction 76

The Integrated Personnel and Pay System – Army (IPPS-A) is a Defense Business 77 System acquisition program under the Milestone Decision Authority of the Army 78 Acquisition Executive. It is managed by the Army Program Executive Office Enterprise 79 Information Systems (PEO EIS), IPPS-A PMO, in Arlington, VA 22202. PEO EIS 80 develops, acquires, and deploys tactical and management Information Technology 81 systems and products and is a Systems Acquisition, Development and Integration 82 Center of Excellence. The IPPS-A mission is to develop and deliver a single, integrated, 83 multi-component Human Resources (HR) and Pay system that streamlines Army HR, 84 enhances the efficiency and accuracy of Army Personnel and Pay procedures, and 85 supports Soldiers and their families. IPPS-A is being deployed incrementally in a series 86 of software releases to the Total U.S. Army Force (Active Army, Army National Guard, 87 and Army Reserves) when complete. A major software release was deployed in 88 January 2023, providing Military Human Resource capabilities and bringing the Total 89 U.S. Army Force into a single authoritative system. This software release serves as the 90 production baseline and is in the Capability Support phase. 91

1.1 Purpose 92

The purpose of the Quality Assurance Surveillance Plan (QASP) is to describe the 93 systematic methods the Government will use to evaluate Contractor actions while 94 performing the requirements as laid out in the Performance Work Statement (PWS) for 95 the Army Military Pay (AMP). The QASP provides a means for evaluating whether the 96 Contractor is meeting the performance standards and acceptable quality levels 97 identified in the PWS and identification of surveillance methods used to document the 98 performance. 99

The QASP has been developed by IPPS-A, PEO EIS to provide direction to personnel 101 performing contract surveillance activities. Personnel performing surveillance of the 102 contract terms and conditions and PWS requirements will periodically review the QASP 103 throughout the life of the contract. The QASP establishes procedures on how this 104 assessment/inspection process will be conducted. It provides details for a 105 continuous oversight process as follows: 106

What will be monitored 107 How monitoring will take place 108 Who will be conduct the monitoring 109 How monitoring efforts and results will be documented 110

The Contractor is responsible for implementing and delivering performance that meets 111 contract standards using its Quality Control Plan. The QASP provides the structure for 112 the Government's surveillance of the Contractor's performance to assure that it meets 113 contract standards. It is the Government's responsibility to be objective, fair and 114 consistent in evaluating Contractor performance. 115

January 12, 2024 2

The QASP is not part of the contract, nor is it intended to duplicate the Contractor's 117 quality control plan. This QASP is a living document. Flexibility in the QASP is required 118 to allow for an increase or decrease in the level of surveillance necessary based on 119 Contractor performance. The Government will provide a copy of the QASP to the 120 Contractor to facilitate open communication. In addition, the QASP should recognize 121 that unforeseen or uncontrollable circumstances might occur that are outside the control 122 of the Contractor. As a result, the QASP should ensure early identification and 123 resolution of performance issues to minimize impact on mission performance. 124

1.2 Intent 126

The intent of this QASP is to: 1) ensure Contractor performance meets or exceeds 127 contract terms, conditions, and specifications; 2) hold the Contractor accountable for 128 quality control; and 3) encourage the Contractor to take appropriate steps to control and 129 improve quality. Accordingly, the Government will perform surveillance on the contract 130 in accordance with this QASP; and reserves the right to monitor the contract in any 131 manner necessary, at any times necessary, and at all places necessary to ensure that 132 the rendered services conform to contract requirements. The Government also reserves 133 the right to perform quality assurance at the Subcontractor level and perform quality 134 assurance at the Contractor's place of business, if applicable. Resolution of non-135 conforming services discovered at the Subcontractor level will be addressed with the 136 prime Contractor. Additionally, the QASP is a living document, and the Government will 137 consider updating the QASP collaboratively with the Contractor post-award and 138 iteratively throughout the period of performance. 139

1.3 Authority 140

Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition 141 Regulation, Inspection of Services clauses, which provides for inspection, acceptance 142 and documentation of the service called for in the contract or order. This acceptance is 143 to be executed by the Contracting Officer or a duly authorized representative. 144

2.0 Contract Description 145

2.1 Background 146

This is a non-personal services contract to obtain Contractor services to develop an 147 Army Military Payroll (AMP) solution using the Oracle PeopleSoft Global Payroll 148 application. The Army launched the Integrated Personnel and Pay System – Army 149 (IPPS-A) Increment II (Inc II) production baseline for Human Resources (HR) 150 processing in December 2023 utilizing PeopleSoft Human Capital Management (HCM). 151 The new AMP capability will be integrated with the IPPS-A production baseline. 152

Development will consider (1) Agile execution combined with Development, Security, 153 and Operations (DevSecOps) for all activities / tasks to allow for Continuous Integration 154 / Continuous Delivery (CI/CD) and compliance with the Continuous Risk Management 155 Framework (cRMF), (2) task automation to reduce / eliminate manual effort, and (3) 156 innovation and creativity in problem-solving. 157

January 12, 2024 3

For this contract, Agile execution is defined as a group of software methods (e.g., 158 Scrum, Kanban, SAFe®) based on iterative and incremental development and 159 deployment of software capabilities for end users, where requirements and solutions 160 evolve through collaboration between cross-functional teams. These teams include the 161 Government and Contractor communities working across parallel environments where 162 capabilities are implemented, and production system capability support is ongoing. 163

Contractors will apply and perform Agile execution in varying degrees to all aspects of 164 the program life cycle across and within key activities. These activities include 165 requirements analysis, elicitation, elaboration, and decomposition; architecture; design; 166 development, implementation; test; integration; training material development; and 167 deployment, in addition to support functions such as program management (planning, 168 reporting), configuration management, change control, release management, quality 169 assurance, and cybersecurity. 170

2.2 Scope of Work 171

The Army requires the following services to deliver an AMP solution as follows: (1) 172 manage, (2) design, configure, build, and test, (3) integrate and test the AMP solution 173 with the IPPS-A production baseline in the integration environment, (4) support AMP 174 release and deployment activities, (5) assess Government software for reusability, (6) 175 develop AMP training material, and (7) transition AMP functionality to production 176 operations. 177

These services will be acquired across the following tasks: 178

Task 1: Program Management / Transition In 179 Task 2: Solution Configuration, Development, and Test 180 Task 3: Solution Integration 181 Task 4: Release and Deployment Support 182 Task 5: Government Software Reuse Integration 183 Task 6: Training Material Development 184 Task 7: Production Operations Transition / Transition Out 185 Task 8: Training Delivery [OPTIONAL] 186

2.3 Type of Contract 187

This is a Hybrid contract including Firm Fixed Price (FFP), Cost Plus Fixed Fee (CPFF) 188 Level of Effort (LOE), CPFF Completion, and Cost Reimbursable (no fee) line items for 189 Travel and Other Direct Costs (ODCs). 190

3.0 Roles and Responsibilities 191

3.1 Government 192

3.1.1 Contracting Officer 193

The Contracting Officer (KO) shall ensure performance of all necessary actions for 194 effective contracting and compliance with the contract terms and shall safeguard the 195

January 12, 2024 4 interests of the United States in the contractual relationship. The KO shall also ensure 196 the Contractor receives impartial, fair, and equitable treatment under this contract. The 197 KO shall determine and document the final assessment of the Contractor's performance 198 in the Government’s past performance tracking system. 199

Assigned CO: Kristin M. Height, Contracting Officer 200 Organization or Agency: U.S. Army Contracting Command – Aberdeen Proving Ground 201 Telephone: (443) 861-4613 202 Email: kristin.m.height.civ@army.mil 203

3.1.2 Contracting Officer’s Representative 204

The Contracting Officer’s Representative (COR) will be identified by separate letter. The 205 COR monitors all technical aspects of the contract and assists in contract 206 administration. The COR is authorized to perform the following functions: assure that 207 the Contractor performs the technical requirements of the contract: perform inspections 208 necessary in connection with contract performance: maintain written and oral 209 communications with the Contractor concerning technical aspects of the contract: issue 210 written interpretations of technical requirements, including Government drawings, 211 designs, specifications: monitor Contractor's performance and notifies both the 212 Contracting Officer and Contractor of any deficiencies; coordinate availability of 213 Government furnished property, and provide site entry of Contractor personnel. A letter 214 of designation issued to the COR, a copy of which is sent to the Contractor, states the 215 responsibilities and limitations of the COR, especially with regard to changes in cost or 216 price, estimates or changes in delivery dates. The COR is not authorized to change any 217 of the terms and conditions of the resulting order. 218

Assigned COR: Dawn Bare, Contracting Officer Representative 219 Organization or Agency: USARMY HQDA IPPS-A 220 Telephone: (703) 545-2743 221 Email: dawn.m.bare2.civ@army.mil 222

3.1.3 Product Manager and Associate PdM 223

The IPPS-A Product Manager (PdM) and AMP Associate PdM (APM) provide primary 224 program oversight. While the PdM and APM may serve as a direct conduit to provide 225 Government guidance and feedback to the Contractor on project matters, the PdM and 226 APM are not empowered to make any contractual commitments or any contract 227 changes on behalf of the Government. The PdM and APM will recommend a COR to 228 the KO, ensure the COR is trained before performing any delegated duties, and will 229 support COR performance assessment activities. 230

Product Manager: LTC Ryan Martin 231 Organization or Agency: USARMY HQDA IPPS-A 232 Telephone: (703) 545-3291 233 Email: ryan.p.martin7.mil@army.mil 234 Associate Product Manager: MAJ Lauri Blair 235 Organization or Agency: USARMY HQDA IPPS-A 236 Telephone: (832) 353-9695 237 Email: lauri.r.blair.mil@army.mil 238

January 12, 2024 5

3.1.4 Performance Monitors 239

3.1.4.1 Functional Monitor 240

The AMP Functional Monitor will provide detailed functional oversight of the 241 Contractor’s performance regarding business value and alignment with the AMP 242 functional requirements and report his or her findings to the COR in a timely, complete, 243 and impartial fashion to support the COR’s administration activities. Any changes that 244 the Contractor deems may affect contract, price, terms, or conditions shall be referred to 245 the Contracting Officer for action. The Functional Monitor is not empowered to provide 246 any guidance to the Contractor or make any contractual commitments or any contract 247 changes on behalf of the Government. 248

Functional Monitor: LTC Erica Miller 249 Organization or Agency: USARMY HQDA ASA MRA 250 Telephone: (703) 614-8208 251 Email: erica.l.miller.mil@army.mil 252

3.1.4.2 Technical Monitors 253

AMP Technical Monitors will provide detailed technical oversight of the Contractor’s 254 performance and report their findings to the COR in a timely, complete, and impartial 255 fashion to support the COR’s administration activities. Any changes that the Contractor 256 deems may affect contract, price, terms, or conditions shall be referred to the 257 Contracting Officer for action. Technical monitors may include performance monitors, 258 inspectors, technical experts, or others who provide information that helps the COR 259 monitor Contractor performance. The Technical Monitors are not empowered to provide 260 any guidance to the Contractor or make any contractual commitments or any contract 261 changes on behalf of the Government. 262

Lead Technical Monitor: Katherine Seifert 263 Organization or Agency: USARMY HQDA IPPS-A 264 Telephone: (571) 314-4046 265 Email: katherine.e.seifert.civ@army.mil 266

3.2 Contractor 267

3.2.1 Contract Manager 268

The Contract Manger manages the full range of acquisition activities. This individual will 269 be the primary point of contact for the KO and any designated CORs. Additionally, the 270 Contract Manger will 1) perform work related to Department of Defense (DoD) Contracts 271 Management, such as FAR, Defense Federal Acquisition Regulation Supplement 272 (DFARS), DoD Instructions (DoDI) 5000 series regulations and Clinger-Cohen Act 273 compliance; 2) track deadlines, deliverables, and other obligations; 3) plan, direct, and 274 manage projects/operations; 4) track and maintain a record of modifications; 4) record 275 and solve contract problems/issues; 5) report on contract deliverables and performance, 276 and 6) ensure product quality. In addition, the Contract Manager will be knowable of 277 different contract types; and will coordinate and interface with the Government 278 contracting office on contract-related action items and management activities. 279

January 12, 2024 6

Contract Manager: TBD 280 Telephone: 281 Email: 282

3.2.2 Program Manager 283

The Contractor will assign an individual to serve as a Program Manager (PM) for this 284 effort. This individual will be the single point of contact for the Government Task Monitor 285 (GTM) and COR. The PM will be responsible for the performance of the work. The 286 name of this person and an alternate, who will act for the Contractor when the PM is 287 absent, will be designated in writing to the Contracting Officer. The Contractor PM and 288 alternate will have full authority to act for the Contractor on all matters relating to daily 289 operation of this contract. 290

Program Manager: TBD 291 Telephone: 292 Email: 293

3.2.3 Quality Manager 294

The Quality Manager will maintain an effective quality control program to ensure 295 services are performed as agreed to in the contract. This will include the following: 296

Updating and implementing procedures to identify, prevent, and ensure the non-297 recurrence of defective services, to include products and software 298

Using programmatic and technical metrics as quality measures to assess 299 performance, progress, speed, cybersecurity, and quality of the product team, 300 leveraging automated tools to the maximum extent practicable 301

Quality Manager: TBD 302 Telephone: 303 Email: 304

4.0 Contract Quality Requirements 305

4.1 Contractor’s Quality Control Program 306

The Contractor’s quality control program (IAW FAR Part 46 and applicable 52.246 307 Clause(s)) is the means by which the Contractor assures that work complies with the 308 requirement of the contract. The Contractor shall ensure services are performed in 309 accordance with the PWS. The Contractor shall identify, prevent, and ensure non-310 recurrence of defective services. 311

4.2 Government Quality Assurance 312

The Government shall evaluate the Contractor’s performance under this contract in 313 accordance with this QASP. This plan is primarily focused on what the Government 314 must do to ensure that the Contractor has performed in accordance with the 315 performance standards. It defines how the performance standards will be applied, the 316 frequency of surveillance and minimum acceptable defect rate(s). 317

January 12, 2024 7

5.0 Government Surveillance 318

5.1 Methods of Surveillance 319

The following Methods of Surveillance will be utilized in monitoring the Contractors’ 320 performance: 321

5.1.1 100% Inspection 322

The Government will inspect 100% of each deliverable or service performed. This 323 surveillance method is appropriate where 100% inspection is required and is practical. 324

5.1.2 Random Sampling 325

The Government will randomly sample less than 100% of a designated deliverable or 326 performed service on a random basis. This surveillance method is appropriate for a 327 deliverable or performed service that occurs infrequently and where 100% inspection is 328 neither required nor practical. 329

5.1.3 Periodic Inspection 330

The Government will inspect less than 100% of a designated deliverable or performed 331 service, on a periodic basis. This surveillance method is appropriate for a deliverable or 332 performed service that occurs infrequently and where 100% inspection is neither 333 required nor practical. 334

5.1.4 Customer Feedback 335

The following procedures will be followed when a complaint is received regarding the 336 Contractor/Contractor Employees. 337

The COR will investigate and determine the validity of the complaint 338

If the complaint is determined to be invalid, the COR will inform the person who 339 submitted the complaint of the reason(s) as soon as practical 340

For validated complaints that are true contractual non-conformances, the 341 procedures in paragraph 6 below will be followed 342

5.1.5 After Action Review (AAR) 343

The Government will perform random AAR’s at the Planning Interval (PI) and/or sprint 344 level up to 100% of PI’s exercised. This surveillance method is appropriate for a 345 deliverable or performed service which will capture best practices, bottlenecks, and 346 future PI/sprint development processes for implementation. 347

5.2 Performance Requirements Summary 348

In evaluating the quality of the Contractor’s performance, the following performance 349 ratings under Table 1 may be used IAW FAR 42.1503, Table 42-1. 350

January 12, 2024 8

Table 1. Evaluation Ratings Definitions 351

Rating Definition Note

Exceptional – Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with few minor problems for which corrective actions taken by the Contractor were highly effective.

To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an Exceptional rating. Also, there should not have been any significant weaknesses identified.

Very Good – Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the Contractor were effective.

To justify a Very Good rating, identify a significant event and state how it was a benefit to the Government. There should not be any significant weaknesses identified.

Satisfactory – Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the Contractor appear or were satisfactory.

To justify a Satisfactory rating, there should have been only minor problems, or major problems the Contractor recovered from without impact to the contract/order.

There should not have been significant weaknesses identified. A fundamental principle of assigning ratings is that Contractors will not be evaluated with a rating lower than Satisfactory solely for not performing beyond the requirements of the contract/order.

Marginal – Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a serious problem for which the Contractor has not yet identified corrective actions. The Contractor proposed actions appear only marginally effective or were not fully implemented.

To justify Marginal performance, identify a significant event in each category that the Contractor had trouble overcoming and state how it impacted the Government. A Marginal rating should be supported by referencing the management tool that notified the Contractor of the contractual deficiency (e.g., management, quality, safety, or environmental deficiency report or letter).

Unsatisfactory – Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which Contractor corrective actions appear or were ineffective.

To justify an Unsatisfactory rating, identify multiple significant events in each category that the Contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating. An Unsatisfactory rating should be supported by referencing the management tools used to notify the Contractor of the contractual deficiencies (e.g., management, quality, safety, or environmental deficiency reports, or letters).

The Performance Requirements Summary or Surveillance Matrix (Table 2) is the list of 352 performance objectives and standards that must be performed by the Contractor. This 353 matrix details the method of surveillance the COR will use to validate and inspect these 354 performance elements. Inspection of each element will be documented in the COR file. 355 Issued Delivery Orders will identify and include all or individually selected metrics from 356 Table 2. The Government will monitor quality measures monthly using agile 357 development tools to the extent available and will formally review quality measures at 358 the end of each PI utilizing agile development tools reporting to the extent available. 359 The Government will consider updating metrics collaboratively with the Contractor post-360 award and iteratively throughout the period of performance. 361

Performance objectives define the desired outcomes. Performance Standards define 362 the level of service required under the contract to successfully meet the performance 363 objective. The inspection methodology defines how, when, and what will be assessed in 364 measuring performance. The Government performs surveillance, using this QASP, to 365 determine the quality of the Contractor's performance as it relates to the performance 366 element standards. The Program Requirements Summary will be used to form the 367 foundation of the COR's inspection checklist. 368

January 12, 2024 9

Table 2. Performance Requirements Summary (Surveillance Matrix) 369

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

Program Management

PWS Section(s):

5.1 - 5.8

Deliverables:

All

Program Planning and Reporting

Quality Control and Management

Risk Management

Schedule Management

Personnel Management

Cost and Software Data Reporting

Contract Funds Management

License Management

Property Management

Knowledge Management

Periodic and ad hoc Meetings and Reporting

RA: Management

Program Management activities span all contract tasks. This includes the management and oversight of all activities performed by Contractor personnel, including Subcontractors, to satisfy the requirements identified in the PWS.

Accurate, complete. and timely support of tasks and functions as directed by the Government Implementation of the schedule Milestones met on time Requirements met within proposed cost Metric reports delivered on time in correct format/content

Meetings and Reporting provides the information needed

Task activities defined in PWS successfully completed

Very Good: meets the following at a minimum of 95%:

Meets milestones as scheduled Meets contract cost baseline Personnel required to perform tasks are qualified and present

Exceptional: meets the following at a minimum of 97%:

Meets milestones as scheduled Meets contract cost baseline Personnel required to perform tasks are qualified and present

100% compliant with all procedures and applicable regulations

90% of Contract Data Requirements List (CDRLs) delivered on time and in compliance with the DD FORM 1423-1

Satisfactory: meets the following at a minimum of 93%:

Meets milestones as scheduled

Meets contract cost baseline

Personnel required to perform tasks are qualified and present

100% Inspection CDRLs Staffing Levels and Quality of Performance

Outcomes and performance of regularly scheduled and ad hoc meetings

Contract Performance

PWS Section(s):

5.1

Deliverables:

A003 A009

Roadmap / Schedule

RA: Schedule

Measures how efficiently the project is progressing compared to the planned timeline

Roadmap / Schedule Performance = Actual number of Scheduled PI Activities and Tasks / Planned number of Scheduled PI Activities and Tasks

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Cost

RA: Cost Control

Measures the cost efficiency of the project by comparing planned with actual costs

Cost Performance = Actual Cost / Planned Cost

Very Good: 95% to <100%

Exceptional: <95%

Satisfactory: 100% 100% Inspection

January 12, 2024 10

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

Small Business Utilization

RA: Small Business Subcontracting

Measures the actual achievements relative to Contractor proposed Small Business Participation Commitment Document by business category

Small Business Utilization = Cumulative percentage total by adding the Percent of total contract value for each business category

Very Good: 26%

Exceptional: 27%

Satisfactory: 25% 100% Inspection

Throughput

PWS Section(s):

5.2 - 5.4

Deliverables:

A003 B006 B012 B014

Story Point Completion Rate (Predictability)

RA: Quality

Measures the rate at which story points are being completed within a sprint and PI

Story Point Completion Rate = the number of story points completed / number of story points planned per sprint/PI

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Story Point Velocity

Measures the number of story points completed versus the Average Velocity per sprint or PI

Average Velocity = total number of story points completed across all sprints or PIs / total number of sprints or PIs

Story Point Velocity = number of Story Points completed in a sprint or PI / Average Velocity per sprint or PI

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Story Completion Rate stories completed in a sprint or PI

Story Completion Rate = number of stories completed / number of stories committed to (agreed upon in the plan) within a sprint or PI

This metric Threshold and Objective percentages may be adjusted if the number of total stories across Agile Teams is less than 20.

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Release Burnup

Measures the amount of work completed for a release (Note: work should increase as sprints are completed until release)

Release Burnup = sum of work completed in sprints associated with a specific release / planned work in the specific release

Very Good: 95% at the end of a specific release

Exceptional: 97% at the end of a specific release

Satisfactory: 93% at the end of a specific release

100% Inspection

January 12, 2024 11

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

Recidivism stories returned to the team

Recidivism = number of stories returned in a PI / number of stories completed in a PI

Very Good: 6%

Exceptional: 4%

Satisfactory: 8% 100% Inspection

Software Quality

PWS Section(s):

5.2 - 5.4

Deliverables:

A003 B014 D001

First-time Pass stories that pass the first time in a sprint or PI

First-time Pass Rate = number of stories passed (completed and accepted) in accordance with Definition of Done in a sprint or PI / number of completed stories in a sprint or PI

Very Good: 95%

Exceptional: 97%

Defect Density defects per PI

Defect Density = number of defective stories in a PI / number of stories completed in a PI

Very Good: 6%

Exceptional: 4%

Satisfactory: 8% 100% Inspection

Automated Testing Code Coverage Rate

Measures how much of the user stories developed or configured are validated by automated testing

Automated Testing Code Coverage Rate = number of user stories developed or configured that are validated by automated testing / total number of user stories developed or configured in a PI

Very Good: 93%

Exceptional: 95%

Satisfactory: 90% 100% Inspection

Escaped Defect Density escaped defects per release to the integration environment

Escaped Defect Density = number of escaped defective stories / total number of stories completed in the release

Very Good: 6%

Exceptional: 4%

Satisfactory: 8% 100% Inspection

Recurring Defects

Measures the percentage of recurring defects at the end of each PI

Total Recurring Defects Percentage = Total Number of Recurring Defects Identified over all PIs / Total Number of Defects Identified over all PIs

Very Good: TBD

Exceptional: TBD

Satisfactory: TBD 100% Inspection

January 12, 2024 12

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

Defect Closure of total defects closed at the end of each PI

Defect Closure Rate = Defects Closed (Identified in a PI or Carried from a previous PI) / Total Defects per PI (Identified in a PI plus Carried from a previous PI)

Very Good: 95%

Exceptional: 97%

Satisfactory: 93% 100% Inspection

Number of Blockers (anything that impedes completion of a story, activity, or work item) events that prohibit completion of a story, activity, or work item

Number of Blockers = Number of blockers currently open that prohibit the completion of a story, activity or work item in a sprint or PI

Very Good: 6 blockers

Exceptional: 4 blockers

Satisfactory: 8 blockers

100% Inspection

Integration (Interoperability) Testing of stories that pass integration testing within the integration environment

Integration (Interoperability) Testing = number of stories that pass integration testing / number of stories included in integration testing

Very Good: 97%

Exceptional: 99%

Satisfactory: 95% 100% Inspection

Delivered Features (or Capabilities)

Measures the delivery progress of each Feature relative to the planned completion rate of each Feature in a PI

Delivered Features (or Capabilities) = % of each Feature (or Capability) completed in a PI / % of each Feature planned in that PI

Very Good: 95%

Exceptional: 97%

Value Delivered

PWS Section(s):

5.2, 5.8

Deliverables:

A003

Delivered Value (Business Value Delivered)

Measures Business Value Delivered in a Sprint / PI

Business Value Delivered = total business value points completed / total business value points planned

Very Good: 93%

Exceptional: 97%

Satisfactory: 90% 100% Inspection

Cybersecurity

PWS Section(s):

5.2, 5.3

Deliverables:

A003 B009 C001

RMF Actions

The Contractor shall follow DoD and Army RMF policies and regulations that govern the Department of Defense (DOD) RMF process (risk identification; risk measurement and assessment; risk mitigation; risk reporting and monitoring; and risk governance)

Very Good: 96%

Exceptional: 100%

January 12, 2024 13

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of

Conduct risk reduction activity by scanning delivered code to identify cybersecurity vulnerabilities and to modify code to reduce / eliminate vulnerabilities.

For vulnerabilities that cannot be eliminated, assist with RMF actions to identify risk, risk measurement and assessment, risk mitigation, risk reporting and monitoring (i.e., POAMs and eMASS documentation)

RMF Actions = Sum of total number of eliminated vulnerabilities and total number of vulnerabilities with POAMs / total number of identified vulnerabilities

Training Material Development

PWS Section(s):

5.6, 5.8

Deliverables:

A003 F001 F002

Training Material Effectiveness

Measures the training materials’ effectiveness to convey the knowledge to users

Accurate and complete Training Materials

Training proficiency levels identified in the agreed to Training Plan are met

Very Good: All training participants achieve a score of 80% or better and an average score of 82%

Exceptional: All training participants achieve a score of 80% or better and an average score of 85%

Satisfactory: All training participants achieve a score of 80% or better and an average score of 80%

100% Inspection of user training scores

Training

PWS Section(s):

5.8

Deliverable:

A003

Training experience satisfaction

Measures training satisfaction using post-training survey. It is used to gauge how satisfied learners are with the training they received.

Training experience satisfaction = sum of all questionnaire scores / number of questionnaire scores

Very Good: 87%

Exceptional: 90%

Satisfactory: 83% 100% Inspection

Transition

PWS Section(s):

5.1, 5.7

Deliverables:

A003 A008 G001

Follow the Transition Plan accepted by the Government and keep the Government fully informed of status throughout the Transition period.

Transition

Planning and Management

Transition Accountability

Transition Activity

Collaboration Asset Transfer

Transfer responsibility for project documentation, resources, assets, and performance to the designated party. Ensure the implementation and readiness of capabilities necessary for all aspects of performance without disruption in schedule, increased costs, degradation to performance, need for increased Government oversight, or likelihood of unsuccessful

Accurate, complete. and timely support of tasks and

Very Good: meets the following at a minimum:

100% compliant with all procedures and applicable regulations

100% of CDRLs delivered on time and in compliance with the DD

FORM 1423-1.

95% Personnel required to perform tasks are qualified and present

Exceptional: meets the following at a minimum:

100% compliant with all procedures and applicable regulations

Satisfactory: meets the following at a minimum:

100% compliant with all procedures and applicable regulations

90% of CDRLs delivered on time and in compliance with the DD

FORM 1423-1.

93% Personnel required to perform tasks are qualified and present

100% Inspection:

CDRLs Outcomes and performance of regularly scheduled and ad hoc meetings

January 12, 2024 14

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of functions as directed by the Government Products delivered on time in correct format/content Meetings and Reporting provides the information needed

Task activities defined in PWS successfully completed

100% of CDRLs delivered on time and in compliance with the DD

FORM 1423-1.

97% Personnel required to perform tasks are qualified and present

AMP

Standards

PWS Section(s):

5.3

Deliverable:

A003 B009 B010 C001

Accuracy of Soldiers that received accurate pay, error free, across processed payroll transactions, such as Soldier data payroll calculation tax calculations rations allotments garnishments Thrift Savings Plan Government liabilities Others

Accuracy = number of Soldiers that receive accurate pay (error free) / number of Soldiers receiving pay

Very Good: Not Applicable

Exceptional: 100%

Satisfactory: 99% 100% Inspection

Timeliness

Measures the time it takes for the system to process payroll transactions for 1M+ payees without impacting day to day operations within the IPPS- A baseline

Very Good: 6 hours

Exceptional: 4 hours

Satisfactory: 8 hours 100% Inspection

Compliance of payroll transactions that comply with LPRDs:

laws policies regulations directives

Compliance = Number of complied with LPRDs / Total number of LPRDs

Very Good: 97%

Exceptional: 99%

Satisfactory: 95% 100% Inspection

Solider Satisfaction

Measures Soldier satisfaction with the payroll process; can be assessed through surveys, feedback forms, or focus groups.

A high level of satisfaction indicates that the system meets the needs of Soldiers and their families.

Soldier Satisfaction = (total survey point score / total achievable points)

Very Good: 95%

Exceptional: 97%

January 12, 2024 15

In addition, the Government will inspect user stories (hereafter stories) at the end of 370 each Sprint to evaluate the “Definition of Done” achieved during the Sprint. The target 371 “Definition of Done” for each Sprint will be defined at the beginning of the Sprint up to 372 and including integration and interoperability criteria. The Government expects that 373 stories for interoperability between applications and interfaces will be explicitly identified 374 as part of the development process. 375

A CPARS rating area (RA) is assigned to each relevant metric. The rating areas are 376 Quality, Management, Schedule, Cost Control, Small Business Subcontracting, 377 regulatory compliance. Each rating area rating will be provided to the COR as input for 378 monthly and annual reporting. 379

The COR, along with the IPPS-A PdM, AMP APM, and designated Government 380 Functional and Technical Monitors, will inspect all tasks to ensure Contractor 381 compliance with the requirements at the conclusion of each Sprint or PI. Delivery will 382 occur by a data pull within the AMP agile development tools repository. Inspection 383 results will be recorded and reported. The COR may find the delivery satisfactory even 384 though further work is required, provided that the specific requirements of the sprint are 385 met. 386

At the conclusion of each sprint, the COR, along with the IPPS-A PdM, AMP APM, and 387 designated Government Functional and Technical Monitors, will review the completed 388 stories and related functionality. Incomplete or inadequate code and stories will be 389 noted in a mutually agreed-upon issue tracker, such as Trello or GitHub Issues, and 390 links to each issue shared with the COR. The Contractor may respond in that tracker as 391 appropriate, addressing the accuracy and validity of the defect as well as any planned 392 corrective action (if not already noted). The Contractor team will discuss and document 393 actions to prevent recurrence in their sprint retrospectives. 394

At the conclusion of the period of performance, a similar procedure will be followed to 395 document discrepancies and to assess overall performance. 396

5.3 Surveillance Schedule 397

A Surveillance Schedule will be created for each month of Contractor performance and 398 filed in the Procurement Integrated Enterprise Environment (PIEE), Surveillance and 399 Performance Monitoring (SPM), Surveillance Documents Section of the Contract File 400 using the Add Smart Form function. The schedule is “Controlled Unclassified 401 Information (CUI)” and is not releasable to anyone other than those authorized by the 402 Government. 403

5.4 Surveillance Instructions/Checklists 404

Surveillance Checklists will be completed when surveillances are conducted and filed in 405 the PIEE, SPM, Surveillance Documents Section of the Contract File using the Add 406 Smart Form function. Supplementary data, such as end of sprint reports directly 407 extracted from the development tools that provide a snapshot of real-time progress will 408 be captured when possible. The objective is to take advantage of any tools and 409 streamlined processes available that capture results suitably to obtain the most 410 accurate results in an administrative-efficient manner. 411

January 12, 2024 16

5.5 Surveillance Documentation 412

No later than the 15th of each month surveillance documentation for the preceding 413 month will be filed in the PIEE, SPM, Surveillance Documents Section. Surveillance 414 documentation includes a COR Status Report, the Surveillance Schedule, all completed 415 Surveillance Checklist(s) for the period the COR Status Report covers, 416 Nonconformance Reports, Performance Assessment Report, Correction Action Report, 417 if any, and Contractor Deliverables, if any. 418

6.0 Nonconformance Reports (NCRs) (Attachment 1) 419

Nonconformance occurs when a Contractor fails to meet Contract or Task Order terms, 420 conditions and/or specifications or the services are defective in some manner. All 421 instances of nonconformance will be documented via a Non-conformance Report 422 Worksheet, Attachment 1 and the Contracting Officer will notify the Contractor of the 423 nonconforming performance via Letter. Resolution of nonconformance discovered at the 424 Subcontractor level will be addressed with the prime Contractor. Nonconformance is 425 classified as either: Level 1 (Minor); Level 2 (Major); or Level 3 (Critical). NCRs may 426 contain multiple nonconformances, each characterized by a level and type. The level of 427 the NCR should normally correspond to the highest level of nonconformance listed in 428 the NCR. For example, if an NCR had one Level 3, one Level 2, and three Level 1 non-429 conformances, the NCR level would be Level 3. The exception might be when the 430 aggregate of multiple nonconformances considered in whole associate to a more 431 serious type of nonconformance defined in FAR 46.101 (e.g., if an NCR had three Level 432 2 nonconformances but the definition of critical nonconformance better describes the 433 impact/effect when considered in whole, the NCR level would be 3). 434

6.1 Level 1 (Minor) 435

Level 1 nonconformances are those that are ‘minor’ in nature such that they can be 436 promptly corrected by the Contractor, present no need for root cause analysis or 437 determination if other contract requirements are affected, present no need for further 438 preventive action, and do not materially reduce the usability of the supplies, services, 439 construction, or deliverables. While the Contractor must correct the nonconformance, 440 further actions and Government follow-up are not required, and no written response 441 from the Contractor is needed. 442

6.2 Level 2 (Major) 443

Level 2 nonconformance(s) are those that are ‘major’ in nature such that they are not 444 promptly correctable, and/or warrant root cause analysis and preventive action, and/or 445 need action by the Contractor to determine if other contract requirements are affected. 446 These nonconformances are serious enough to warrant not only correction of the 447 nonconformance, but also a written response from the Contractor in the form of a 448 Corrective Action Plan (CAP), and Government follow-up. The qualifying distinction 449 between a Level 1 and Level 2 nonconformance is the need for a CAP and Government 450 follow-up. An NCR identifying a Level 2 nonconformance may also be issued after 451 repetitive Level 1 instances of the same or similar nonconformance demonstrate the 452

January 12, 2024 17

Contractor is failing to institute…