IPPS-A AMP QASP - DRAFT - 21 NOV 2023.pdf

CUI

Integrated Personnel and Pay System – Army

(IPPS-A)

Army Military Payroll

Quality Assurance Surveillance Plan

(QASP)

DRAFT

W91CRB-24-R-0003

November 21, 2023

Prepared by:

IPPS-A Product Manager (PdM) Future Capabilities Office

IPPS-A AMP QASP - DRAFT CUI

November 21, 2023 ii

THIS PAGE INTENTIONALLY LEFT BLANK 14

November 21, 2023 iii

Table of Contents 18

1.0 Introduction ......................................................................................................... 1 19

1.1 Purpose ............................................................................................................... 1 20

1.2 Intent .................................................................................................................... 2 21

1.3 Authority.............................................................................................................. 2 22

2.0 Contract Description .......................................................................................... 2 23

2.1 Background ......................................................................................................... 2 24

2.2 Scope of Work .................................................................................................... 3 25

2.3 Type of Contract ................................................................................................. 3 26

3.0 Roles and Responsibilities ................................................................................ 3 27

3.1 Government ........................................................................................................ 3 28

3.1.1 Contracting Officer ................................................................................... 3 29

3.1.2 Contracting Officer’s Representative ........................................................ 4 30

3.1.3 Product Manager and Associate PdM ...................................................... 4 31

3.1.4 Performance Monitors .............................................................................. 5 32

3.2 Contractor ........................................................................................................... 5 33

3.2.1 Contract Manager ..................................................................................... 5 34

3.2.2 Program Manager .................................................................................... 6 35

3.2.3 Quality Manager ....................................................................................... 6 36

4.0 Contract Quality Requirements ......................................................................... 6 37

4.1 Contractor’s Quality Control Program .............................................................. 6 38

4.2 Government Quality Assurance ........................................................................ 6 39

5.0 Government Surveillance .................................................................................. 7 40

5.1 Methods of Surveillance .................................................................................... 7 41

5.1.1 100% Inspection ....................................................................................... 7 42

5.1.2 Random Sampling .................................................................................... 7 43

5.1.3 Periodic Inspection ................................................................................... 7 44

5.1.4 Customer Feedback ................................................................................. 7 45

5.1.5 After Action Review (AAR) ....................................................................... 7 46

5.2 Performance Requirements Summary ............................................................. 7 47

5.3 Surveillance Schedule ..................................................................................... 15 48

November 21, 2023 iv

5.4 Surveillance Instructions/Checklists .............................................................. 15 49

5.5 Surveillance Documentation ........................................................................... 15 50

6.0 Nonconformance Reports (NCRs) (Attachment 1) ........................................ 16 51

6.1 Level 1 (Minor) .................................................................................................. 16 52

6.2 Level 2 (Major) .................................................................................................. 16 53

6.3 Level 3 (Critical) ................................................................................................ 17 54

6.4 Documenting NCRs .......................................................................................... 17 55

6.5 Tracking and Resolving NCRs ........................................................................ 18 56

6.6 Corrective Action Report (CAR). (Attachment 2) ........................................... 18 57

6.7 Performance Assessment Report (PAR). (Attachment 3) ............................. 18 58

7.0 Deduct Plan ....................................................................................................... 18 59

8.0 Data Analysis .................................................................................................... 19 60

9.0 Acceptance of Services ................................................................................... 20 61

10.0 COR Status Report ........................................................................................... 20 62

11.0 Contractor Manpower Reporting Initiative ..................................................... 20 63

12.0 Contractor Performance Assessment Reporting System (CPARS) ............. 20 64

13.0 Combating Trafficking In Persons (CTIP) ....................................................... 20 65

14.0 COR/Quality Assurance Surveillance Files .................................................... 21 66

15.0 QASP Changes ................................................................................................. 21 67

Attachment 1 - Nonconformance Report Worksheet ............................................... 22 68

Attachment 2 - Corrective Action Report (CAR) ....................................................... 23 69

Attachment 3 - Performance Assessment Report (PAR) ......................................... 24 70

List of Tables 73

Table 1. Evaluation Ratings Definitions ........................................................................... 8 74

Table 2. Performance Requirements Summary (Surveillance Matrix) ............................. 9 75

November 21, 2023 1

1.0 Introduction 78

The Integrated Personnel and Pay System – Army (IPPS-A) is a Defense Business 79 System acquisition program under the Milestone Decision Authority of the Army 80 Acquisition Executive. It is managed by the Army Program Executive Office Enterprise 81 Information Systems (PEO EIS), IPPS-A PMO, in Arlington, VA 22202. PEO EIS 82 develops, acquires, and deploys tactical and management Information Technology 83 systems and products and is a Systems Acquisition, Development and Integration 84 Center of Excellence. The IPPS-A mission is to develop and deliver a single, integrated, 85 multi-component Human Resources (HR) and Pay system that streamlines Army HR, 86 enhances the efficiency and accuracy of Army Personnel and Pay procedures, and 87 supports Soldiers and their families. IPPS-A is being deployed incrementally in a series 88 of software releases to the Total U.S. Army Force (Active Army, Army National Guard, 89 and Army Reserves) when complete. A major software release was deployed in 90 January 2023, providing Military Human Resource capabilities and bringing the Total 91 U.S. Army Force into a single authoritative system. This software release serves as the 92 production baseline and is in the Capability Support phase. 93

1.1 Purpose 94

The purpose of the Quality Assurance Surveillance Plan (QASP) is to describe the 95 systematic methods the Government will use to evaluate Contractor actions while 96 performing the requirements as laid out in the Performance Work Statement (PWS) for 97 the Army Military Pay (AMP). The QASP provides a means for evaluating whether the 98 Contractor is meeting the performance standards and acceptable quality levels 99 identified in the PWS and identification of surveillance methods used to document the 100 performance. 101

The QASP has been developed by IPPS-A, PEO EIS to provide direction to personnel 103 performing contract surveillance activities. Personnel performing surveillance of the 104 contract terms and conditions and PWS requirements will periodically review the QASP 105 throughout the life of the contract. The QASP establishes procedures on how this 106 assessment/inspection process will be conducted. It provides details for a 107 continuous oversight process as follows: 108

What will be monitored 109 How monitoring will take place 110 Who will be conduct the monitoring 111 How monitoring efforts and results will be documented 112

The Contractor is responsible for implementing and delivering performance that meets 113 contract standards using its Quality Control Plan. The QASP provides the structure for 114 the Government's surveillance of the Contractor's performance to assure that it meets 115 contract standards. It is the Government's responsibility to be objective, fair and 116 consistent in evaluating Contractor performance. 117

November 21, 2023 2

The QASP is not part of the contract, nor is it intended to duplicate the Contractor's 119 quality control plan. This QASP is a living document. Flexibility in the QASP is required 120 to allow for an increase or decrease in the level of surveillance necessary based on 121 Contractor performance. The Government will provide a copy of the QASP to the 122 Contractor to facilitate open communication. In addition, the QASP should recognize 123 that unforeseen or uncontrollable circumstances might occur that are outside the control 124 of the Contractor. As a result, the QASP should ensure early identification and 125 resolution of performance issues to minimize impact on mission performance. 126

1.2 Intent 128

The intent of this QASP is to: 1) ensure Contractor performance meets or exceeds 129 contract terms, conditions, and specifications; 2) hold the Contractor accountable for 130 quality control; and 3) encourage the Contractor to take appropriate steps to control and 131 improve quality. Accordingly, the Government will perform surveillance on the contract 132 in accordance with this QASP; and reserves the right to monitor the contract in any 133 manner necessary, at any times necessary, and at all places necessary to ensure that 134 the rendered services conform to contract requirements. The Government also reserves 135 the right to perform quality assurance at the Subcontractor level and perform quality 136 assurance at the Contractor's place of business, if applicable. Resolution of non-137 conforming services discovered at the Subcontractor level will be addressed with the 138 prime Contractor. 139

1.3 Authority 140

Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition 141 Regulation, Inspection of Services clauses, which provides for inspection, acceptance 142 and documentation of the service called for in the contract or order. This acceptance is 143 to be executed by the Contracting Officer or a duly authorized representative. 144

2.0 Contract Description 145

2.1 Background 146

This is a non-personal services contract to obtain Contractor services to develop an 147 Army Military Payroll (AMP) solution using the Oracle PeopleSoft Global Payroll 148 application. The Army launched the Integrated Personnel and Pay System – Army 149 (IPPS-A) Increment II (Inc II) production baseline for Human Resources (HR) 150 processing in December 2023 utilizing PeopleSoft Human Capital Management (HCM). 151 The new AMP capability will be integrated with the IPPS-A production baseline. 152

Development will consider (1) Agile execution combined with Development, Security, 153 and Operations (DevSecOps) for all activities / tasks to allow for Continuous Integration 154 / Continuous Delivery (CI/CD) and compliance with the Continuous Risk Management 155 Framework (cRMF), (2) task automation to reduce / eliminate manual effort, and (3) 156 innovation and creativity in problem-solving. 157

November 21, 2023 3

For this contract, Agile execution is defined as a group of software methods (e.g., 158 Scrum, Kanban, SAFe®) based on iterative and incremental development and 159 deployment of software capabilities for end users, where requirements and solutions 160 evolve through collaboration between cross-functional teams. These teams include the 161 Government and Contractor communities working across parallel environments where 162 capabilities are implemented, and production system capability support is ongoing. 163

Contractors will apply and perform Agile execution in varying degrees to all aspects of 164 the program life cycle across and within key activities. These activities include 165 requirements analysis, elicitation, elaboration, and decomposition; architecture; design; 166 development, implementation; test; integration; training material development; and 167 deployment, in addition to support functions such as program management (planning, 168 reporting), configuration management, change control, release management, quality 169 assurance, and cybersecurity. 170

2.2 Scope of Work 171

The Army requires the following services, and the Contractor shall deliver an AMP 172 solution as follows: (1) design, configure, build, and test, (2) integrate and test the AMP 173 solution with the IPPS-A production baseline, (3) develop AMP training material, (4) 174 support AMP release and deployment activities, and (5) transition AMP functionality to 175 capability support. 176

These services will be acquired across the following tasks: 177

Task 1: Program Management 178 Task 2: Solution Configuration, Development, and Test 179 Task 3: Solution Integration 180 Task 4: Release and Deployment Support 181 Task 5: Government Software Reuse Integration 182 Task 6: Training Material Development 183 Task 7: Capability Support Transition 184 Task 8: Training Delivery [OPTIONAL] 185

2.3 Type of Contract 186

This is a Hybrid contract including Firm Fixed Price (FFP), Cost Plus Fixed Fee (CPFF) 187 Level of Effort (LOE), CPFF Completion, and Cost Reimbursable (no fee) line items for 188 Travel and Other Direct Costs (ODCs). 189

3.0 Roles and Responsibilities 190

3.1 Government 191

3.1.1 Contracting Officer 192

The Contracting Officer (KO) shall ensure performance of all necessary actions for 194 effective contracting and compliance with the contract terms and shall safeguard the 195

November 21, 2023 4 interests of the United States in the contractual relationship. The KO shall also ensure 196 the Contractor receives impartial, fair, and equitable treatment under this contract. The 197 KO shall determine and document the final assessment of the Contractor's performance 198 in the Government’s past performance tracking system. 199

Assigned CO: Kristin M. Height, Contracting Officer 200 Organization or Agency: U.S. Army Contracting Command – Aberdeen Proving Ground 201 Telephone: (443) 861-4613 202 Email: kristin.m.height.civ@army.mil 203

3.1.2 Contracting Officer’s Representative 204

The Contracting Officer’s Representative (COR) will be identified by separate letter. The 205 COR monitors all technical aspects of the contract and assists in contract 206 administration. The COR is authorized to perform the following functions: assure that 207 the Contractor performs the technical requirements of the contract: perform inspections 208 necessary in connection with contract performance: maintain written and oral 209 communications with the Contractor concerning technical aspects of the contract: issue 210 written interpretations of technical requirements, including Government drawings, 211 designs, specifications: monitor Contractor's performance and notifies both the 212 Contracting Officer and Contractor of any deficiencies; coordinate availability of 213 Government furnished property, and provide site entry of Contractor personnel. A letter 214 of designation issued to the COR, a copy of which is sent to the Contractor, states the 215 responsibilities and limitations of the COR, especially with regard to changes in cost or 216 price, estimates or changes in delivery dates. The COR is not authorized to change any 217 of the terms and conditions of the resulting order. 218

Assigned COR: Dawn Bare, Contracting Officer Representative 219 Organization or Agency: USARMY HQDA IPPS-A 220 Telephone: (703) 545-2743 221 Email: dawn.m.bare2.civ@army.mil 222

3.1.3 Product Manager and Associate PdM 223

The IPPS-A Product Manager (PdM) and AMP Associate PdM (APM) provide primary 224 program oversight. While the PdM and APM may serve as a direct conduit to provide 225 Government guidance and feedback to the Contractor on project matters, the PdM and 226 APM are not empowered to make any contractual commitments or any contract 227 changes on behalf of the Government. The PdM and APM will recommend a COR to 228 the KO, ensure the COR is trained before performing any delegated duties, and will 229 support COR performance assessment activities. 230

Product Manager: LTC Ryan Martin 231 Organization or Agency: USARMY HQDA IPPS-A 232 Telephone: (703) 545-3291 233 Email: ryan.p.martin7.mil@army.mil 234 Associate Product Manager: MAJ Lauri Blair 235 Organization or Agency: USARMY HQDA IPPS-A 236 Telephone: (832) 353-9695 237 Email: lauri.r.blair.mil@army.mil 238

November 21, 2023 5

3.1.4 Performance Monitors 239

3.1.4.1 Functional Monitor 240

The AMP Functional Monitor will provide detailed functional oversight of the 241 Contractor’s performance regarding business value and alignment with the AMP 242 functional requirements and report his or her findings to the COR in a timely, complete, 243 and impartial fashion to support the COR’s administration activities. Any changes that 244 the Contractor deems may affect contract, price, terms, or conditions shall be referred to 245 the Contracting Officer for action. The Functional Monitor is not empowered to provide 246 any guidance to the Contractor or make any contractual commitments or any contract 247 changes on behalf of the Government. 248

Functional Monitor: LTC Erica Miller 249 Organization or Agency: USARMY HQDA ASA MRA 250 Telephone: (703) 614-8208 251 Email: erica.l.miller.mil@army.mil 252

3.1.4.2 Technical Monitors 253

AMP Technical Monitors will provide detailed technical oversight of the Contractor’s 254 performance and report their findings to the COR in a timely, complete, and impartial 255 fashion to support the COR’s administration activities. Any changes that the Contractor 256 deems may affect contract, price, terms, or conditions shall be referred to the 257 Contracting Officer for action. Technical monitors may include performance monitors, 258 inspectors, technical experts, or others who provide information that helps the COR 259 monitor Contractor performance. The Technical Monitors are not empowered to provide 260 any guidance to the Contractor or make any contractual commitments or any contract 261 changes on behalf of the Government. 262

Lead Technical Monitor: Katherine Seifert 263 Organization or Agency: USARMY HQDA IPPS-A 264 Telephone: (571) 314-4046 265 Email: katherine.e.seifert.civ@army.mil 266

3.2 Contractor 267

3.2.1 Contract Manager 268

The Contract Manger manages the full range of acquisition activities. This individual will 269 be the primary point of contact for the KO and any designated CORs. Additionally, the 270 Contract Manger will 1) perform work related to Department of Defense (DoD) Contracts 271 Management, such as FAR, Defense Federal Acquisition Regulation Supplement 272 (DFARS), DoD Instructions (DoDI) 5000 series regulations and Clinger-Cohen Act 273 compliance; 2) track deadlines, deliverables, and other obligations; 3) plan, direct, and 274 manage projects/operations; 4) track and maintain a record of modifications; 4) record 275 and solve contract problems/issues; 5) report on contract deliverables and performance, 276 and 6) ensure product quality. In addition, the Contract Manager will be knowable of 277 different contract types; and will coordinate and interface with the Government 278 contracting office on contract-related action items and management activities. 279

November 21, 2023 6

Contract Manager: TBD 280 Telephone: 281 Email: 282

3.2.2 Program Manager 283

The Contractor will assign an individual to serve as a Program Manager (PM) for this 284 effort. This individual will be the single point of contact for the Government Task Monitor 285 (GTM) and COR. The PM will be responsible for the performance of the work. The 286 name of this person and an alternate, who will act for the Contractor when the PM is 287 absent, will be designated in writing to the Contracting Officer. The Contractor PM and 288 alternate will have full authority to act for the Contractor on all matters relating to daily 289 operation of this contract. 290

Program Manager: TBD 291 Telephone: 292 Email: 293

3.2.3 Quality Manager 294

The Quality Manager will maintain an effective quality control program to ensure 295 services are performed as agreed to in the contract. This will include the following: 296

Updating and implementing procedures to identify, prevent, and ensure the non-297 recurrence of defective services, to include products and software 298

Using programmatic and technical metrics as quality measures to assess 299 performance, progress, speed, cybersecurity, and quality of the product team, 300 leveraging automated tools to the maximum extent practicable 301

Quality Manager: TBD 302 Telephone: 303 Email: 304

4.0 Contract Quality Requirements 305

4.1 Contractor’s Quality Control Program 306

The Contractor’s quality control program (IAW FAR Part 46 and applicable 52.246 307 Clause(s)) is the means by which the Contractor assures that work complies with the 308 requirement of the contract. The Contractor shall ensure services are performed in 309 accordance with the PWS. The Contractor shall identify, prevent, and ensure non-310 recurrence of defective services. 311

4.2 Government Quality Assurance 312

The Government shall evaluate the Contractor’s performance under this contract in 313 accordance with this QASP. This plan is primarily focused on what the Government 314 must do to ensure that the Contractor has performed in accordance with the 315 performance standards. It defines how the performance standards will be applied, the 316 frequency of surveillance and minimum acceptable defect rate(s). 317

November 21, 2023 7

5.0 Government Surveillance 318

5.1 Methods of Surveillance 319

The following Methods of Surveillance will be utilized in monitoring the Contractors’ 320 performance: 321

5.1.1 100% Inspection 322

The Government will inspect 100% of each deliverable or service performed. This 323 surveillance method is appropriate where 100% inspection is required and is practical. 324

5.1.2 Random Sampling 325

The Government will randomly sample less than 100% of a designated deliverable or 326 performed service on a random basis. This surveillance method is appropriate for a 327 deliverable or performed service that occurs infrequently and where 100% inspection is 328 neither required nor practical. 329

5.1.3 Periodic Inspection 330

The Government will inspect less than 100% of a designated deliverable or performed 331 service, on a periodic basis. This surveillance method is appropriate for a deliverable or 332 performed service that occurs infrequently and where 100% inspection is neither 333 required nor practical. 334

5.1.4 Customer Feedback 335

The following procedures will be followed when a complaint is received regarding the 336 Contractor/Contractor Employees. 337

The COR will investigate and determine the validity of the complaint 338

If the complaint is determined to be invalid, the COR will inform the person who 339 submitted the complaint of the reason(s) as soon as practical 340

For validated complaints that are true contractual non-conformances, the 341 procedures in paragraph 6 below will be followed 342

5.1.5 After Action Review (AAR) 343

The Government will perform random AAR’s at the Planning Interval (PI) and/or sprint 344 level up to 100% of PI’s exercised. This surveillance method is appropriate for a 345 deliverable or performed service which will capture best practices, bottlenecks, and 346 future PI/sprint development processes for implementation. 347

5.2 Performance Requirements Summary 348

In evaluating the quality of the Contractor’s performance, the following performance 349 ratings under Table 1 may be used IAW FAR 42.1503, Table 42-1. 350

November 21, 2023 8

Table 1. Evaluation Ratings Definitions 351

Rating Definition Note

Exceptional – Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with few minor problems for which corrective actions taken by the Contractor were highly effective.

To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an Exceptional rating. Also, there should not have been any significant weaknesses identified.

Very Good – Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being evaluated was accomplished with some minor problems for which corrective actions taken by the Contractor were effective.

To justify a Very Good rating, identify a significant event and state how it was a benefit to the Government. There should not be any significant weaknesses identified.

Satisfactory – Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the Contractor appear or were satisfactory.

To justify a Satisfactory rating, there should have been only minor problems, or major problems the Contractor recovered from without impact to the contract/order.

There should not have been significant weaknesses identified. A fundamental principle of assigning ratings is that Contractors will not be evaluated with a rating lower than Satisfactory solely for not performing beyond the requirements of the contract/order.

Marginal – Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being evaluated reflects a serious problem for which the Contractor has not yet identified corrective actions. The Contractor proposed actions appear only marginally effective or were not fully implemented.

To justify Marginal performance, identify a significant event in each category that the Contractor had trouble overcoming and state how it impacted the Government. A Marginal rating should be supported by referencing the management tool that notified the Contractor of the contractual deficiency (e.g., management, quality, safety, or environmental deficiency report or letter).

Unsatisfactory – Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which Contractor corrective actions appear or were ineffective.

To justify an Unsatisfactory rating, identify multiple significant events in each category that the Contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating. An Unsatisfactory rating should be supported by referencing the management tools used to notify the Contractor of the contractual deficiencies (e.g., management, quality, safety, or environmental deficiency reports, or letters).

The Performance Requirements Summary or Surveillance Matrix (Table 2) is the list of 352 performance objectives and standards that must be performed by the Contractor. This 353 matrix details the method of surveillance the COR will use to validate and inspect these 354 performance elements. Inspection of each element will be documented in the COR file. 355

Performance objectives define the desired outcomes. Performance Standards define 356 the level of service required under the contract to successfully meet the performance 357 objective. The inspection methodology defines how, when, and what will be assessed in 358 measuring performance. The Government performs surveillance, using this QASP, to 359 determine the quality of the Contractor's performance as it relates to the performance 360 element standards. The Program Requirements Summary will be used to form the 361 foundation of the COR's inspection checklist. 362

November 21, 2023 9

Table 2. Performance Requirements Summary (Surveillance Matrix) 363

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

Program Management

PWS Section

5.1 - 5.8

Deliverables:

All

Program Planning and Reporting

Quality Control and Management

Risk Management

Schedule Management

Personnel Management

Cost and Software Data Reporting

Contract Funds Management

License Management

Property Management

Knowledge Management

Periodic and ad hoc Meetings and Reporting

RA: Management

Program Management activities span all contract tasks. This includes the management and oversight of all activities performed by Contractor personnel, including Subcontractors, to satisfy the requirements identified in the PWS.

Accurate, complete. and timely support of tasks and functions as directed by the Government

Implementation of the schedule

Milestones met on time Requirements met within proposed cost Metric reports delivered on time in correct format/content

Meetings and Reporting provides the information needed

Task activities defined in PWS successfully completed

Very Good: meets the following at a minimum of 95%:

Meets milestones as scheduled Meets contract cost baseline Personnel required to perform tasks are qualified and present

Exceptional: meets the following at a minimum of 97%:

Meets milestones as scheduled Meets contract cost baseline Personnel required to perform tasks are qualified and present

100% compliant with all procedures and applicable regulations

90% of Contract Data Requirements List (CDRLs) delivered on time and in compliance with the DD FORM 1423-1

Satisfactory: meets the following at a minimum of 93%:

Meets milestones as scheduled

Meets contract cost baseline

Personnel required to perform tasks are qualified and present

100% Inspection

CDRLs Staffing Levels and Quality of Performance

Outcomes and performance of regularly scheduled and ad hoc meetings

Progress against Roadmap

RA: Schedule

Compares targeted versus delivered value to help determine if appropriate progress is being made. This informs future planning, prioritization, and investment.

Very Good: 95% of Delivered values

Exceptional: 97% of Delivered values

Satisfactory: 93% of Delivered values

100% Inspection

Contract Performance

Schedule

Index (SPI)

Measures how efficiently the project is progressing compared to the planned timeline (SPI = Earned Value / Planned Value)

Very Good: 95% SPI

Exceptional: 97% SPI

Satisfactory: 93%

SPI

100% Inspection

November 21, 2023 10

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

5.1

Deliverables:

A003 A009

Cost

Index (CPI)

RA: Cost Control

Evaluates the cost efficiency of the project by comparing earned value with actual costs (CPI = Earned Value / Actual Cost)

Very Good: 95% CPI

Exceptional: 97% CPI

Satisfactory: 93%

CPI

100% Inspection

Customer Satisfaction

Measures the level of satisfaction of stakeholders with the Contractor's performance

Customer satisfaction score = (total survey point score / total questions) x 100

Very Good: 95% Customer Satisfaction Score

Exceptional: 97% Customer Satisfaction Score

Satisfactory: 93% Customer Satisfaction Score

100% Inspection

Small Business Utilization

RA: Small Business Subcontracting

Actual achievements relative to Contractor proposed Small Business Participation Commitment Document by business category

Cumulative percentage total by adding the Percent of total contract value for each business category

Very Good: 26% Total cumulative percentage

Exceptional: 27% Total cumulative percentage

Satisfactory: 25% Total cumulative percentage

100% Inspection

% of Small business subcontracting value

RA: Small Business Subcontracting

Total value of small business contracting incurred on contract.

Total value of small business contracting = (Total value of small business contracting / Total value of contracting)

Very Good: 26% Total value of business contracting

Exceptional: 27% Total value of business contracting SPI Customer Satisfaction Score

Satisfactory: 25% Total value of business contracting

Agile Process

5.2 - 5.4

Deliverables:

A003 B006 B012 B014

Story Points

RA: Quality

Measure the complexity of a story and total amount of work to be completed in a sprint

% of Story Points completed versus planned

Very Good: 95%

Exceptional: 97%

% of Story Points completed versus planned

Satisfactory: 93%

Velocity

Actual amount of work completed in a sprint

Planned story points versus velocity (avg story point per sprint) each sprint

Very Good: 95%

Exceptional: 97%

Planned story points versus velocity (avg story point per sprint) each sprint

Satisfactory: 93%

100% Inspection

Velocity Predictability

Average Velocity is a measurement of the average amount of story (or story points)

Used to calculate Velocity

Calculated at the end of Planning Intervals (PIs)

Used to calculate Velocity

Calculated at the end of PIs

November 21, 2023 11

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold delivered across a given time period.

Average Velocity = Total number of story (or story points) delivered / Total number of Sprints

Story Completion Rate

Number of stories completed in a sprint (and release)

This metric Threshold and Objective percentages may be adjusted if the number of total stories across Agile Teams is less then 20.

% of stories completed versus planned

Very Good: 95%

Exceptional: 97%

% of stories completed versus planned

Satisfactory: 93%

Release Burnup

Amount of work completed for a release (Note: work should increase as sprints are completed until release)

% of release work completed versus planned

Very Good: 95%

Exceptional: 97%

% of release work completed versus planned

Satisfactory: 93%

Recidivism

Stories returned to the team (this is where the levels of “why” come in)

% of stories returned versus completed in a sprint

Very Good: 6%

Exceptional: 4%

% of stories returned versus completed in a sprint

Satisfactory: 8%

Agile Quality

5.2 - 5.4

Deliverables:

A003 D001

First-time Pass Rate

Number of stories that pass the first time

% of stories passed (completed and accepted) in accordance with Definition of Done versus completed in a sprint

Very Good: 95%

Exceptional: 97%

% of stories passed (completed and accepted) in accordance with Definition of Done versus completed a sprint

Satisfactory: 93%

Defect Count

Number of defects per spring, PI, and release

% of defects per lines of code

Very Good: 6%

Exceptional: 4%

% of defects per lines of code

Satisfactory: 8%

Test Coverage

Level of testing performed in the value stream process

% of test coverage of capabilities, features, epics, and/or stories

Very Good: 95%

Exceptional: 97%

% of test coverage of capabilities, features, epics, and/or user stories

Satisfactory: 93%

Periodic Inspection

Number of Blockers

Number of events that prohibit completion of an activity or work item

Number of contract blockers currently open that prohibit the completion

Number of contract blockers currently open that prohibit the completion of an

November 21, 2023 12

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold of an activity or work item in a sprint or release

Very Good: 6 blockers

Exceptional: 4 blockers activity or work item in a sprint or release

Satisfactory: 8 blockers

Integration (Interoperability) Testing

Number of stories that pass integration testing

% of stories that pass integration testing versus planned

Very Good: 97%

Exceptional: 99%

% of stories that pass integration testing versus planned

Satisfactory: 95%

100% Inspection

Delivered Features (or Capabilities)

Delivery progress relative to outcome (Note: Features (or Capabilities) should increase as PIs are completed until release)

% of Delivered Features (or Capabilities) completed in accordance with Definition of Done versus planned

Very Good: 95%

Exceptional: 97%

% of Delivered Features (or Capabilities) completed in accordance with Definition of Done versus planned

Satisfactory: 93%

Capability Delivery

PWS Section 5.2, 5.8

Deliverables:

A003 B004 B009 B010

Delivered Value Points

Value points delivered to users for a release (Note: Is are completed until release)

% of Value delivered versus Value planned

Very Good: 95%

Exceptional: 97%

8% of Value delivered versus Value planned

Satisfactory: 93%

100% Inspection

Level of User

User satisfaction based on the value delivered in a release

% of User satisfaction based on the delivered value in a release

Very Good: 95%

Exceptional: 97%

% of User satisfaction based on the delivered value in a release

Satisfactory: 93%

100% Inspection of Releases with a random number of users surveyed

Cybersecurity

5.2, 5.3

Deliverables:

A003 B009 C001

RMF actions

The Contractor shall follow DoD and Army RMF policies and regulations that govern the Department of Defense (DOD) RMF process (risk identification; risk measurement and assessment; risk mitigation; risk reporting and monitoring; and risk governance)

Conduct risk reduction activity by scanning delivered code to identify cybersecurity vulnerabilities and to modify code to reduce/eliminate/ vulnerabilities.

For vulnerabilities that cannot be eliminated, assist with RMF actions to identify risk, risk measurement and assessment, risk mitigation, risk reporting and monitoring (i.e., POAMs and eMASS documentation)

Very Good: Not Applicable

Exceptional: 100%

Satisfactory: 99.8% 100% Inspection

November 21, 2023 13

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold

Method of Surveillance

Training Material

Development

5.6, 5.8

Deliverables:

A003 F001 F002

Training Material Effectiveness

Measure of the training materials’ effectiveness to convey the knowledge to users.

Accurate and complete Training Materials

Training proficiency levels identified in the agreed to Training Plan are met

Very Good: All training participants achieve a score of 80% or better and an average score of 82%

Exceptional: All training participants achieve a score of 80% or better and an average score of 85%

Accurate and complete Training Materials

Training proficiency levels identified in the agreed to Training Plan are met

Satisfactory: All training participants achieve a score of 80% or better and an average score of 80%

100% Inspection of user training scores

Training

5.8

Deliverable:

A003

Training experience satisfaction

Training metrics to measure and can usually be found in a post-training survey. It is used to gauge how satisfied learners are with the training they received.

Very Good: 87% Score from User Questionnaire

Exceptional: 90% Score from User Questionnaire

Satisfactory: 83% Score from User Questionnaire

Transition

5.1, 5.7

Deliverables:

A003 A008 G001

Follow the Transition Plan accepted by the Government and keep the Government fully informed of status throughout the Transition period.

Transition Planning and Management

Transition Accountability

Transition Activity

Collaboration Asset Transfer

Transfer responsibility for project documentation, resources, assets, and performance to the designated party.

Ensure the implementation and readiness of capabilities necessary for all aspects of performance without disruption in schedule, increased costs, degradation to performance, need for increased Government oversight, or likelihood of unsuccessful

Accurate, complete. and timely support of tasks and functions as directed by the Government

Products delivered on time in correct format/content

Meetings and Reporting provides

Very Good: meets the following at a minimum:

100% compliant with all procedures and applicable regulations

100% of CDRLs delivered on time and in compliance with the DD

FORM 1423-1.

95% Personnel required to perform tasks are qualified and present

Exceptional: meets the following at a minimum:

100% compliant with all procedures and applicable regulations

100% of CDRLs delivered on time and in compliance with the DD

FORM 1423-1.

97% Personnel required to perform tasks are qualified and present

Satisfactory: meets the following at a minimum:

100% compliant with all procedures and applicable regulations

90% of CDRLs delivered on time and in compliance with the DD

FORM 1423-1.

93% Personnel required to perform tasks are qualified and present

100% Inspection:

CDRLs Outcomes and performance of regularly scheduled and ad hoc meetings

November 21, 2023 14

Type

Deliverable or Required

Services

Rating Area

Description

Performance Standard(s)

Objective

Acceptable Quality Level

(AQL)

Threshold the information needed

Task activities defined in PWS successfully completed

AMP

Standards

5.3

Deliverable:

A003 B009 B010 C001

Accuracy

Percentage of accurately processed payroll transactions

Identify errors in:

Soldier data payroll calculation tax calculations rations allotments garnishments Thrift Savings Plan Government liabilities others

Soldiers receives accurate pay, error free:

Very Good: Not Applicable

Exceptional: 100%

Soldiers receives accurate pay, error free:

Satisfactory: 99%

Timeliness

Measures the time it takes for the system to process payroll transactions for 1M+ payees

Very Good: 6 hours

Exceptional: 4 hours

Satisfactory: 8 hours 100% Inspection

Compliance

Measures the percentage of payroll transactions that comply with:

relevant laws regulations policies

Compliance Score

Very Good: 95%

Exceptional: 97%

Compliance Score

Satisfactory: 93%

100% Inspection

Solider

Measures Soldier satisfaction with the payroll process; can be assessed through surveys, feedback forms, or focus groups.

A high level of satisfaction indicates that the system meets the needs of Soldiers and their families.

Soldier satisfaction score = (total survey point score / total questions) x 100

Very Good: 95%

Exceptional: 97%

Soldier satisfaction score = (total survey point score / total questions) x 100

Satisfactory: 93%

100% Inspection

*Note, the Government notifies the Contractor that it intends to adopt automated testing and will establish metric(s) for automated 364 testing at a later date under the terms of QASP paragraph 15.0. 365

In addition, the Government will inspect user stories (hereafter stories) at the end of 366 each Sprint to evaluate the “Definition of Done” achieved during the Sprint. The target 367 “Definition of Done” for each Sprint will be defined at the beginning of the Sprint up to 368 and including integration and interoperability criteria. The Government expects that 369 stories for interoperability between applications and interfaces will be explicitly identified 370 as part of the development process. 371

November 21, 2023 15

A CPARS rating area (RA) is assigned to each relevant metric. The rating areas are 372 Quality, Management, Schedule, Cost Control, Small Business Subcontracting, 373 regulatory compliance. Each rating area rating will be provided to the COR as input for 374 monthly and annual reporting. 375

The COR, along with the IPPS-A PdM, AMP APM, and designated Government 376 Functional and Technical Monitors, will inspect all tasks to ensure Contractor 377 compliance with the requirements at the conclusion of each Sprint or PI. Delivery will 378 occur by a data pull within the AMP agile development tools repository. Inspection 379 results will be recorded and reported. The COR may find the delivery satisfactory even 380 though further work is required, provided that the specific requirements of the sprint are 381 met. 382

At the conclusion of each sprint, the COR, along with the IPPS-A PdM, AMP APM, and 383 designated Government Functional and Technical Monitors, will review the completed 384 stories and related functionality. Incomplete or inadequate code and stories will be 385 noted in a mutually agreed-upon issue tracker, such as Trello or GitHub Issues, and 386 links to each issue shared with the COR. The Contractor may respond in that tracker as 387 appropriate, addressing the accuracy and validity of the defect as well as any planned 388 corrective action (if not already noted). The Contractor team will discuss and document 389 actions to prevent recurrence in their sprint retrospectives. 390

At the conclusion of the period of performance, a similar procedure will be followed to 391 document discrepancies and to assess overall performance. 392

5.3 Surveillance Schedule 393

A Surveillance Schedule will be created for each month of Contractor performance and 394 filed in the Procurement Integrated Enterprise Environment (PIEE), Surveillance and 395 Performance Monitoring (SPM), Surveillance Documents Section of the Contract File 396 using the Add Smart Form function. The schedule is “Controlled Unclassified 397 Information (CUI)” and is not releasable to anyone other than those authorized by the 398 Government. 399

5.4 Surveillance Instructions/Checklists 400

Surveillance Checklists will be completed when surveillances are conducted and filed in 401 the PIEE, SPM, Surveillance Documents Section of the Contract File using the Add 402 Smart Form function. Supplementary data, such as end of sprint reports directly 403 extracted from the development tools that provide a snapshot of real-time progress will 404 be captured when possible. The objective is to take advantage of any tools and 405 streamlined processes available that capture results suitably to obtain the most 406 accurate results in an administrative-efficient manner. 407

5.5 Surveillance Documentation 408

No later than the 15th of each month surveillance documentation for the preceding 409 month will be filed in the PIEE, SPM, Surveillance Documents Section. Surveillance 410 documentation includes a COR Status Report, the Surveillance Schedule, all completed 411 Surveillance Checklist(s) for the period the COR Status Report covers, 412

November 21, 2023 16

Nonconformance Reports, Performance Assessment Report, Correction Action Report, 413 if any, and Contractor Deliverables, if any. 414

6.0 Nonconformance Reports (NCRs) (Attachment 1) 415

Nonconformance occurs when a Contractor fails to meet Contract or Task Order terms, 416 conditions and/or specifications or the services are defective in some manner. All 417 instances of nonconformance will be documented via a Non-conformance Report 418 Worksheet, Attachment 1 and the Contracting Officer will notify the Contractor of the 419 nonconforming performance via Letter. Resolution of nonconformance discovered at the 420 Subcontractor level will be addressed with the prime Contractor. Nonconformance is 421 classified as either: Level 1 (Minor); Level 2 (Major); or Level 3 (Critical). NCRs may 422 contain multiple nonconformances, each characterized by a level and type. The level of 423 the NCR should normally correspond to the highest level of nonconformance listed in 424 the NCR. For example, if an NCR had one Level 3, one Level 2, and three Level 1 non-425 conformances, the NCR level would be Level 3. The exception might be when the 426 aggregate of multiple nonconformances considered in whole associate to a more 427 serious type of nonconformance defined in FAR 46.101 (e.g., if an NCR had three Level 428 2 nonconformances but the definition of critical nonconformance better describes the 429 impact/effect when considered in whole, the NCR level would be 3). 430

6.1 Level 1 (Minor) 431

Level 1 nonconformances are those that are ‘minor’ in nature such that they can be 432 promptly corrected by the Contractor, present no need for root cause analysis or 433 determination if other contract requirements are affected, present no need for further 434 preventive action, and do not materially reduce the usability of the supplies, services, 435 construction, or deliverables. While the Contractor must correct the nonconformance, 436 further actions and Government follow-up are not required, and no written response 437 from the Contractor is needed. 438

6.2 Level 2 (Major) 439

Level 2 nonconformance(s) are those that are ‘major’ in nature such that they are not 440 promptly correctable, and/or warrant root cause analysis and preventive action, and/or 441 need action by the Contractor to determine if other contract requirements are affected. 442 These nonconformances are serious enough to warrant not only correction of the 443 nonconformance, but also a written response from the Contractor in the form of a 444 Corrective Action Plan (CAP), and Government follow-up. The qualifying distinction 445 between a Level 1 and Level 2 nonconformance is the need for a CAP and Government 446 follow-up. An NCR identifying a Level 2 nonconformance may also be issued after 447 repetitive Level 1 instances of the same or similar nonconformance demonstrate the 448 Contractor is failing to institute effective corrective action and/or failing to implement 449 effective quality and/or internal control(s).