Attachment J.4 QASP.pdf

Attachment J.4 – QASP March 11, 2022

QUALITY ASSURANCE SURVEILLANCE PLAN

(QASP)

1.0 PURPOSE

1.1 This QASP is a Government-developed and applied document used to make sure the systematic Quality Assurance (QA) methods are used in the administration of this contract. The intent is to ensure that the Contractor performs in accordance with the performance metrics and the Government receives the quality of services called for in the contract.

1.2 The purpose of the QASP is to describe the systematic methods used to monitor performance and to identify the required documentation and the resources to be employed. The QASP provides a means for evaluating whether the Contractor is meeting the performance standards/quality levels identified in the Statement of Work (SOW) and the Contractor’s Quality Control Plan (QCP), and to ensure that the Government pays only for the level of services received.

1.3 This QASP defines the roles and responsibilities of all members of the Integrated Project Team, identifies the performance objectives, defines the methodologies used to monitor and evaluate the Contractor’s performance, describes QA documentation requirements, and describes the analysis of QA monitoring results.

2.0 AUTHORITY

Authority for issuance of this QASP is provided under Contract Section E – Inspection and Acceptance which provides for inspections and acceptance of the services and deliverables called for in service contracts to be executed by the Contracting Officer of a duly authorized representative.

3.0 PERFORMANCE MANAGEMENT APPROACH AND STRATEGY

3.1 The SOW structures the acquisition around “what” service or quality level is required, as opposed to “how” the Contractor should perform the work (i.e., results, not compliance). This QASP will define the performance management approach taken to monitor and manage the Contractor’s performance to ensure the expected outcomes or performance objectives communicated in the SOW are achieved. Performance management rests on developing a capability to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management; this analysis yields information that indicates whether expected outcomes for the project are being achieved by the Contractor.

3.2 Performance management represents a significant shift from the more traditional QA concepts in several ways. Performance management focuses on assessing whether outcomes are being achieved and to what extent. This approach migrates away from scrutiny of compliance with the processes and practices used to achieve the outcome. A performance-based approach enables the Contractor to play a large role in how the work is performed, as long as the proposed processes are within the stated constraints. The only exceptions to process reviews are those required by law (federal, state, and local) and compelling business situations, such as safety and health. A “results” focus provides the Contractor flexibility to continuously improve and innovate over the course of the Contract as long as the critical outcomes expected are being achieved and/or the desired performance levels are being met.

3.3 The Contractor is responsible for the quality of all work performed. The Contractor measures that quality through the Contractor’s own QCP. Quality control is work output, not workers, and therefore includes all work performed under this Order regardless of whether the work is performed by Contractor employees or by Sub-contractors. The Contractor’s QCP will set forth the staffing and procedures for self-inspecting the quality, timeliness, responsiveness, customer satisfaction, and other performance requirements in the SOW. The Contractor will develop and implement a performance management system with processes to assess and report its performance to the designated Government representative. This QASP enables the Government to take advantage of the Contractor’s QCP.

3.4 The Government will assess performance using Contractor Performance Assessment Reporting System (CPARS) to determine how the Contractor is performing against communicated performance objectives. The CPAR assesses a Contractor’s performance, both positive and negative, and provides a record on a given contract during a specified period of time. More information pertaining to CPARS can be found at:

http://www.cpars.csd.disa.mil/cparsfiles/pdfs/DoD-CPARS-Guide.pdf. Each assessment will be based on objective data (or measurable, subjective data when objective data is not available) supportable by program and contract management data. CPAR performance expectations will be addressed in the Government and Contractor’s initial post-award meeting. Potential sources of data may include, but are not limited to, the following:

• Status and progress reviews

• Production and management reviews

• Management and engineering process reviews (e.g. risk management, requirements management, etc.)

• Cost performance reports and other cost and schedule metrics

• Other program measures and metrics such as:

Measures of progress and status of resources Measures of deliverable timeliness and accuracy Measures of product quality and process performance Fleet and sponsor feedback/comments and satisfaction ratings

• Systems engineering and other technical progress reviews

• Technical interchange meetings

• Physical and functional configuration audits

• Quality reviews and quality assurance evaluations

• Functional performance evaluations

• Subcontract Reports

3.5 A preliminary CPARS evaluation/rating will be accomplished as noted in Paragraph 5.0 below. The purpose of this review is to determine whether the Contractor is performing at least at a Satisfactory level for each area to be assessed using Table 3. Further, the formal CPARS ratings are used as reference material by others in source selection.

4.0 ROLES AND RESPONSIBILITIES

4.1 Procuring Contracting Officer (PCO)

4.1.1 An individual duly appointed with the authority to enter into, (PCO), or administer, (Administrative Contracting Officer (ACO)), contracts and make related determination and findings on behalf of the Government. The PCO for this contract is identified in Section G, Ddl- G10 Government Contract Administration Points-of-Contact and Responsibilities. The ACO will be designated in the resulting Order. Contracting Officers are designated via a written warrant, which sets forth limitations of their respective authority.

4.1.2 The Contracting Officer ensures performance of all necessary actions for effective contract administration, ensures compliance with the terms of the contract and safeguards the interests of the United States in the contractual relationship. It is the Contracting Officer that ensures the Contractor receives impartial, fair and equitable treatment under the Order. The Contracting Officer is ultimately responsible for the final determination of the acceptability of the Contractor’s performance.

4.2 Contract Specialist

4.2.1 Assigned by the PCO to provide daily administration of the contract.

4.2.2 Provides input to the PCO and the COR as to the quality of performance for areas addressed in this QASP.

4.3 Contracting Officer’s Representative (COR)

4.3.1 An individual appointed in writing by the PCO to act as their authorized representative to assist in technical administration of the Order. The COR is appointed in the contract award. The limitations of authority are contained in a written letter of appointment which is a formal Attachment to the contract.

4.3.2 The COR is responsible for technical administration of the Contract and assures proper Government surveillance of the Contractor’s technical performance. The COR provides QASP reports to the PCO.

4.3.3 The COR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf. Any changes that the Contractor deems may affect contract price, terms, or conditions shall be referred to the Contracting Officer for action.

4.4 Subject Matter Expert (SME)

4.4.1 SMEs are individuals who may be assigned by the COR to perform limited technical oversight of specific projects, work areas, or Technical Instructions issued under the contract.

4.4.2 The SME provides input to the COR as to the quality of technical performance for their respective area(s) of expertise.

4.4.3 A Government SME cannot, in any manner, alter the scope of the contract, make commitments or authorize any changes on the Government’s behalf.

5.0 SCHEDULE

The initial QASP evaluation will be finalized no later than the end of month eight (8) of the Base Period. In order to accomplish this, the following schedule applies:

5.1 Contractor Self-Assessment (written) due to the Contracting Officer and the COR no later than the end of month seven (7) of the Period of Performance based on six (6) months of support.

Failure of the Contractor to make a timely delivery will be viewed as the Contractor’s overall inability to comply with Contract schedules.

5.2 COR Written Assessment due to the Contracting Officer no later than the end of week two

(2) of month eight (8) of the Period of Performance.

5.3 Week four (4) of month eight (8) of the Period of Performance. The Contracting Officer will hold a meeting with the COR and the Contractor for the purpose of reviewing inputs and determining the overall assessment level for the period.

5.4 The QASP evaluation and steps described in 5.1, 5.2 and 5.3 will be accomplished on an annual basis.

6.0 IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY

LEVELS

6.1 The required performance standards and quality levels are included in Table (1), “Performance Standards”. If the Contractor meets the required service or performance level, the Contractor will receive positive preliminary CPARS ratings. If the Contractor fails to meet the required performance level, the result will be negative CPARS ratings.

7.0 METHODOLOGIES TO MONITOR PERFORMANCE

7.1 Surveillance Techniques

In an effort to minimize the performance management burden, simplified surveillance methods shall be used by the Government to evaluate Contractor performance when appropriate. The COR will use the following methods of surveillance:

Customer Feedback

7.2 Customer Feedback

7.2.1 The Contractor is expected to establish and maintain professional communication between its employees and customers. The primary objective of this communication is customer satisfaction. Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through customer complaints.

7.2.2 Performance management drives the Contractor to be customer focused through initially and internally addressing customer complaints and investigating the issues and/or problems but the customer always has the option to communicate complaints to the PCO, as opposed to the Contractor.

7.2.3 Customer complaints, to be considered valid, must set forth clearly and in writing the detailed nature of the complaint, must be signed, and must be forwarded to the COR. The COR will accept those customer complaints, investigate and work with the PCO and Contractor to resolve the issue.

7.2.4 Customer feedback may also be obtained either from the results of formal customer satisfaction surveys or from random customer complaints.

7.3 Acceptable Quality Levels

7.3.1The Acceptable Quality Levels (AQLs) included in Table 3 for Contractor performance are structured to allow the Contractor to manage how the work is performed.

8.0 QUALITY ASSURANCE DOCUMENTATION

8.1 The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes. Performance standards and performance monitoring techniques are expressed in Table 3.

8.2 Monitoring System

The Government’s QA surveillance, accomplished by the COR, in conjunction with the PCO, will be reported using preliminary CPARS. Formal CPARS evaluations shall be conducted on an annual basis. Preliminary CPARS evaluations will be accomplished in accordance with Paragraph 5.0 Schedule.

The tables below set forth the performance ratings, standards, outcome, and surveillance methods to document the results of the surveillance as well as interim and informal reviews (i.e., IPRs) on a more frequent basis.

(a) Performance Ratings: The Government will evaluate the Contractor’s performance of the Performance Work Statement (PWS) and the Contracting Officer will assign one of the following ratings:

(1) Satisfactory

(2) Unsatisfactory

The standards associated with these ratings are given in the following Table 1:

Table 1: Overall Performance Ratings

Overall Performance Rating Standard Satisfactory A minimum of “Satisfactory” ratings for all performance evaluation criteria.

Unsatisfactory A rating of “Unsatisfactory” for one or more performance evaluation criteria.

(b) Objectives/Determination

1. Interim/Informal

Interim/Informal performance evaluations will be provided to assess performance upon request.

2. Yearly Review

The PCO will make a performance determination at the end of each evaluation period.

The determination will be based upon the COR’s recommendations, the Contractor’s comments, including any Self-Evaluation Report, and any other information deemed relevant by the PCO. The PCO shall resolve disagreements between the COR’s recommendations and the Contractor’s comments/report regarding the evaluation. The PCO’s performance determination is unilateral and final. The PCO will document the determination and provide a copy to the Contractor within 30 days of receipt of the Contractor’s self-assessment.

Table 2: Objectives

Assessment Period Acceptable

Performance Definition

How Measured Outcome

Base Year Option Year 1 Option Year 2 Option Year 3 Option Year 4

All measurement areas rated at least

“Satisfactory”

Every 12 months using the QASP covering the previous 12 months

(+) Meet the acceptable performance definition

(-) Does not meet the acceptable performance definition

(a) Performance Evaluation Criteria. The Contractor’s performance will be evaluated using the criteria and standards provided for each objective and identified in Table 3 of this contract clause.

(b) Organization. The performance evaluation organization consists of the PCO, who will serve as the Determining Official, and the COR. In some instances, a SME will be assigned to the contract in addition to the COR.

(c) This performance evaluation does not replace any other requirement for evaluating

Contractor performance that may be required by this contract such as a CPARS report.

TABLE 3: PERFORMANCE STANDARDS

PERFORMANCE

OBJECTIVE

PERFORMANCE

STANDARD

ACCEPTABLE

QUALITY

LEVELS

METHODS OF

SURVEILLANCE

Deliverable Timeliness Data and other deliverables received 100% on schedule.

Data deliverables received at least 95% on schedule

Contractor self-reporting in Monthly Progress Report (MPR), random inspection to validate

Deliverable Quality Data Deliverables received are well researched, complete and technically accurate. Rework is not required. Other deliverables meet all Contract requirements.

Data deliverables received with a first pass yield at least 95% of the time.

Other deliverable items are delivered such that no ship schedules are adversely impacted.

Contractor’s MPR, Government review and approval process;

Feedback from Fleet

PERFORMANCE

STANDARD

ACCEPTABLE

QUALITY

LEVELS

METHODS OF

SURVEILLANCE

Staffing Contractor provides qualified personnel at levels required.

Lapses in coverage may occasionally occur and are managed without any adverse impact on performance.

New/and or substitute resumes submitted IAW Contract requirements.

Personnel work products fully consistent with resume qualifications.

Contractor provides required staffing levels at least 95% of the time. It takes no more than 2 mos. to fully staff all required positions with fully qualified personnel.

Contractor complies with all Contract requirements for resume approval 100% of the time.

Personnel work products are consistent with resume qualifications.

COR review of resumes submitted, personnel work products, and Contractor MPR.

Other Direct Costs

(ODC)

ODCs are reported and invoiced accurately and on time. Contractor complies with contract requirements for ODC authorization 100% of the time. Burdened costs are no higher than proposed.

Contractor complies with contract requirements for ODC authorization 100% of the time.

Burdened costs are no higher than 5% more than proposed.

Contractor’s MPR.

COR review of costs incurred and invoices.

Management Performance

Contractor management ensures timely performance, problem resolution, responsiveness to customer requirements, and effective communications with Government and other stakeholders.

Problems are resolved quickly with minimal Government involvement;

management is responsive to Government requests/concerns;

communication is routinely effective and timely.

COR / Specialist / PCO involvement

PERFORMANCE

STANDARD

ACCEPTABLE

QUALITY

LEVELS

METHODS OF

SURVEILLANCE

Costs and Invoicing Contractor routinely completes the effort within the originally agreed-to estimated cost. Funds and resources are used in a cost-effective manner.

Cost reports are timely, accurate, complete and clearly written.

Invoices are timely (no more than 3 weeks after expenses are incurred).

Contractor is fully compliant with the performance standards by month 3 of performance.

Review of MPR and invoices.

Combatting Trafficking in Persons FAR 52.222- 50.

The Contractor has informed the Contracting Officer and the agency Inspector General 100% of the time of any credible information it has received that alleges a Contractor employee, Subcontractor, Sub-contractor employee, or their agent has engaged in conduct that violates the policy.

100% compliance of notification of incidents as required by Paragraph (d) of the clause.

The Contractor is in 100% compliance of maintaining a compliance plan (including annual certification) as required by Paragraph (h) of the clause.

SW Defect Removal Efficiency.

(When SW development services are invoked)

Defect Removal Efficiency = (I/ I+E) *100 where I = Pre-delivery (Internal) errors including review defects as well as testing defects and E = Post-delivery

Defect Removal Efficiency >= 85%

Contractor self-reporting in Monthly Progress Report (MPR), random inspection to validate

PERFORMANCE

STANDARD

ACCEPTABLE

QUALITY

LEVELS

METHODS OF

SURVEILLANCE

defects (External) or defects found after the release.

The performance standard is Defect Removal Efficiency >=90%.

The performance standard and acceptable quality level will be reviewed/adjusted by technical instruction, when the Software Development services are invoked.

Defect Density Defined as the Defects found/Size (actual number of Story Points).

The performance standard is <=2 defects per 50 story points.

The performance standard and acceptable quality level will be reviewed/adjusted by technical instruction, when the Software Development services are invoked.

<= 5 defects/50 story points

Contractor self-reporting in Monthly Progress Report

Number of Escaped Defects

Defined as the number of bugs that are not caught in testing and end up in product release. The performance standard is <=2 Escaped Defects.

The performance standard and

<= 5 Escaped Defects Contractor self-reporting in Monthly Progress Report

PERFORMANCE

STANDARD

ACCEPTABLE

QUALITY

LEVELS

METHODS OF

SURVEILLANCE

acceptable quality level will be reviewed/adjusted by technical instruction, when the Software Development services are invoked.