Attachment 1 Quality Assurance Surveillance Plan (QASP).docx

CUI

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

1. Contract or Task Order Title HC1013-25-R-0008: Integrated Defense Enterprise Acquisition System (IDEAS) – Tier 2 / Tier 3 Support

2. Purpose:

This QASP provides a systematic method to evaluate performance for the stated contract. This QASP explains the following:

· What will be monitored?

· How monitoring will take place.

· Who will conduct the monitoring?

· How monitoring efforts and results will be documented.

This QASP does not detail how the contractor accomplishes the work. Rather, the QASP is created with the premise that the contractor is responsible for management and quality control actions to meet the terms of the contract. It is the Government’s responsibility to be objective, fair, and consistent in evaluating performance. In addition, the QASP should recognize that unforeseen and uncontrollable situations may occur.

This QASP is a “living document” and the Government may review and revise it on a regular basis. Updates shall ensure that the QASP remains a valid, useful, and enforceable document. Copies of the original QASP and revisions shall be provided to the contractor and Government officials implementing surveillance activities.

3. Roles and Responsibilities:

The following personnel shall oversee and coordinate surveillance activities.

Program/Project Manager (PM) – The PM provides program oversight and supports the COR’s performance assessment activities.

Contracting Officer (KO) – The KO shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The KO shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. Determine the final assessment of the contractor’s performance.

Contracting Officer’s Representative (COR) – The COR is responsible for providing continuous technical oversight of the contractor’s performance. The COR uses the QASP to conduct the oversight/surveillance process. The COR shall keep a Quality Assurance file that accurately documents the contractor’s actual performance. The purpose is to ensure that the contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the KO. The COR is required to provide an annual performance assessment to the KO which will be used in documenting past performance. The QASP is the primary tool for documenting contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government’s behalf.

4. Primary Method of Surveillance:

· 100 Percent Inspection. This is usually only the most appropriate method for infrequent tasks or tasks with stringent performance requirements, e.g., where safety or health is a concern. With this method, performance is inspected/evaluated at each occurrence. One hundred percent inspection is too expensive to be used in most cases.

· Random Sampling. This is usually the most appropriate method for recurring tasks. With random sampling, services are sampled to determine if the level of performance is acceptable. Random sampling works best when the number of instances of the services being performed is very large and a statistically valid sample can be obtained. Computer programs may be available to assist in establishing sampling procedures.

· Periodic Inspection. These services are monitored weekly, monthly, quarterly, semiannually, annually, etc. Periodic types of activities are perfect for periodic inspection because not only are they infrequent, but there is normally a predetermined, specified time frame within which the tasks must be accomplished. Therefore, you know exactly when to conduct the evaluations. Periodic inspections automatically become 100 percent evaluations or "100 percent checks." Inspections should be divided and scheduled by frequency: annual, semiannual, quarterly, monthly, weekly and as required. Sometimes services are required for which the time or frequency cannot be predicted, such as accident investigations, one-time special tasking by higher headquarters, etc. These would be labeled "as required inspections." Others are known and predictable such as the quarterly status report or the monthly travel report currently included in some DISA service contracts.

Surveillance Matrix

The Surveillance Matrix (Appendix 1) is the list of performance objectives and standards that must be performed by the contractor. This matrix details the method of surveillance the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file.

Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. The Government performance surveillance, using this QASP, to determine the quality of the contractor’s performance as it relates to the performance element standards. The AQL performance Standard Table should be used to form the foundation of the COR’s inspection checklist.

5. Performance Standards:

The Performance Standards are captured in the table in Appendix 1.

6. Documenting Performance:

Acceptable Performance.

The Government shall document positive performance. A Performance Assessment Report (PAR) template is attached (Appendix 2). Any report may become part of the supporting documentation for fixed price payments, award/incentive fee payments, Contractor Performance Assessment Reporting System (CPARS) entries, or other actions. Government acceptance of services permits the contractor to invoice for the service fee.

Unacceptable performance.

Unacceptable contractor performance can be identified by customer complaints (Appendix 3) or upon schedule surveillance evaluations. When unacceptable performance occurs, the COR shall inform the contractor. This will normally be in writing unless circumstances necessitate verbal communication and may be through the PAR template (Appendix 2). The COR shall document the discussion and place it in the COR file and provide a copy to the KO.

When the COR determines formal written communication is required, the COR shall prepare a Corrective Action Report (CAR). The CAR is forwarded to the KO for dissemination to the contractor. A CAR template is attached to this QASP (Appendix 4).

The contractor shall acknowledge receipt of the CAR in writing. The CAR will specify if the contractor is required to prepare a corrective action plan to document how the contractor shall correct the unacceptable performance and avoid a recurrence. The CAR will also state how long after receipt the contractor has to present this corrective action plan to the KO and COR. The Government may withhold payment for services until corrective action is taken. The Government shall review the contractor's corrective action plan to determine acceptability. Upon Government acceptance of the corrected services the contractor will be permitted to invoice for the service fee.

Any CAR may become a part of the supporting documentation for contract payment deductions, fixed fee deductions, award fee nonpayment, CPARS or other actions deemed necessary by the KO.

7. Mandatory or regulatory compliance items:

The contractor will provide initial proof of compliance with regulatory and compliance items contained in the contract. The Contracting Officer's Representative (COR) and the Contracting Officer (KO) will evaluate the initial submission and will conduct ad hoc reviews of the contractor's compliance throughout the remainder of the contract. The contractor will monitor compliance as required by FAR 52.222-50, Combating Trafficking in Persons, and report any issues to the KO. The contractor shall inform its employees in writing of contractor employee whistleblower rights and protections as required by DFARS 252.203-7002, Requirement to Inform Employees of Whistleblower Rights. The contractor shall also ensure that the clause is included in all subcontracts.

8. Acceptable Quality Level (AQL):

The AQLs are captured in the table in Appendix 1.

9. Evaluation Method:

The evaluation methodology for task areas on this contract is contained within the table in Appendix 1.

10. Performance Rating

In evaluating the quality of contractor’s performance, the following performance ratings may be used (same as CPAR ratings).

Appendix 1 - Surveillance Matrix

Performance Standards Acceptable Quality Levels Methods of Calculation

Artifacts are 95% UAT passed and are 95% accurate with less than 5% bugs requiring rework or hotfixes.

Minimum Acceptable:

> Artifacts are 90% UAT passed and are 90% accurate with less than 10% bugs requiring rework or hotfixes.

Code will include no libraries, repositories or other external dependencies that DISA doesn’t maintain intellectual property (IP) for.

Calculation:

Number of “pass” results received from PSD.

All non-technical UAT tests must have 3 testers unless permission is obtained from the Government.

Measurement Interval –

A UAT is conducted every 3 weeks, except during EOY processing.

Measurement Period – The UAT will last 2 days, except after EOY processing where it will last 3 days.

Source of Measurement Data – Data will be collected from emails from testers; a final concurrence email from the Government test lead.

Method of Surveillance – Periodic inspection. The Government will conduct UAT every 2-6 weeks. Testers will be Government employees assigned by the customer (PSD); pass/fail results will be determined by the tester.

Timing of Measurement – Measurements are taken within five (5) days of the end of the Measurement Period.

Reporting Frequency: - Reporting to commence upon Contract Award and continue throughout the Period of Performance.

Monthly reporting of Service Level Agreement (SLA) and / or Quality Assurance Surveillance Plan (QASP) requirements.

Exceptional > 95% Very Good > 90% Satisfactory = 90% Marginal = 80-90% Unsatisfactory < 80%

IDEAS software is up to date within six months and no more than one minor version behind throughout the Period of Performance. DI software is up to the latest Major version within six months and stays up to date within three months of each update.

Minimum Acceptable:

IDEAS software is up to date within six months and no more than 2 minor versions behind throughout the Period of Performance. DI software is up to the latest Major version within six months and stays up to date within five months of each update.

Calculation:

Number of deliverables provided at the required or requested time.

Measurement Interval –

Monthly data collection beginning at the first day of the month.

Measurement Period – The full month, regardless of the number of days in the month.

Source of Measurement Data – Data will be collected from deliverables provided by the Contractor as well as the timing of the required deliverables. It may also be randomly collected from appropriate Government systems or other DISA directorates.

Method of Surveillance – Periodic inspection and random sampling. Data will be collected from deliverables provided by the Contractor as well as the timing of the required deliverables. It may also randomly collect reports from appropriate Government systems or other DISA directorates.

Timing of Measurement – Measurements are taken within five (5) business days of the delivery due date.

Reporting Frequency: - Reporting to commence upon Contract Award and continue throughout the Period of Performance.

Monthly reporting of SLA and / or QASP requirements.

Exceptional < 4 months Very Good = 4-5 months Satisfactory = within 6 months Marginal = within 7-8 months Unsatisfactory > 8 months

Tier 2 Resolution/Escalation to Tier 3

Target Criteria:

Critical <= 30 Minutes High <= 4 Hours Medium <= 8 Hours Low <= 5 Calendar Days

Minimum Acceptable:

Critical <= 60 Minutes High <= 6 Hours Medium <= 10 Hours Low <= 5 Calendar Days Calculation:

Response time relative to severity level.

Measurement Interval –

Monthly data collection beginning at the first day of the month.

Measurement Period – The full month, regardless of the number of days in the month.

Source of Measurement Data – Data will be collected from reports provided by the Contractor. It may also be randomly collected from ITSM+ or other appropriate Government systems.

Method of Surveillance – Periodic inspection and random sampling. The Government will obtain data from delivered reports monthly. It may also randomly collect reports from ITSM+ or other appropriate Government systems.

Timing of Measurement – Measurements are taken within five (5) days of the end of the Measurement Period.

Reporting Frequency: - Reporting to commence upon Contract Award and continue throughout the Period of Performance.

Monthly reporting of SLA and / or QASP requirements.

Exceptional = Critical <= 20 Minutes High <= 2 Hours Medium <= 4 Hours Low <= 1 Calendar Day

Very Good = Critical <= 30 Minutes High <= 4 Hours Medium <= 8 Hours Low <= 3 Calendar Days

Satisfactory = Critical <= 60 Minutes High <= 6 Hours Medium <= 10 Hours Low <= 5 Calendar Days

Marginal = Critical <= 90 Minutes High <= 8 Hours Medium <= 12 Hours Low <= 7 Calendar Days

Unsatisfactory = Critical <= 120 Minutes High <= 10 Hours Medium <= 14 Hours Low <= 9 Calendar Days

Tier 3 Resolution/Escalation to External Support Team

Target Criteria:

Critical <= 30 Minutes High <= 4 Hours Medium <= 8 Hours Low <= 3 Calendar Days

Minimum Acceptable:

Critical <= 60 Minutes High <= 6 Hours Medium <= 10 Hours Low <= 5 Calendar Days Calculation:

Response time relative to severity level. If a full resolution cannot be implemented in the time allotted, then a workaround is acceptable until a full resolution can be implemented.

Measurement Interval –

Monthly data collection beginning at the first day of the month.

Measurement Period – The full month, regardless of the number of days in the month.

Source of Measurement Data – Data will be collected from reports provided by the Contractor. It may also be randomly collected from ITSM+ or other appropriate Government systems.

Method of Surveillance – Periodic inspection and random sampling. The Government will obtain data from delivered reports monthly. It may also randomly collect reports from ITSM+ or other appropriate Government systems.

Timing of Measurement – Measurements are taken within five (5) days of the end of the Measurement Period.

Reporting Frequency: - Reporting to commence upon Contract Award and continue throughout the Period of Performance.

Monthly reporting of SLA and / or QASP requirements.

Exceptional = Critical <= 20 Minutes High <= 2 Hours Medium <= 4 Hours Low <= 1 Calendar Day

Very Good = Critical <= 30 Minutes High <= 4 Hours Medium <= 8 Hours Low <= 3 Calendar Days

Satisfactory = Critical <= 60 Minutes High <= 6 Hours Medium <= 10 Hours Low <= 5 Calendar Days

Marginal = Critical <= 90 Minutes High <= 8 Hours Medium <= 12 Hours Low <= 7 Calendar Days

Unsatisfactory = Critical <= 120 Minutes High <= 10 Hours Medium <= 14 Hours Low <= 9 Calendar Days

Tier 2 and Tier 3 Ticket Management

Target Criteria:

100% of unresolved tickets are updated weekly. All open, unresolved tickets will be updated weekly in TSM/Remedy. The updates shall include status and progression.

Minimum Acceptable:

95% of unresolved tickets are updated weekly. All open, unresolved tickets will be updated weekly in TSM/Remedy. The updates shall include status and progression.

Calculation:

Number of tickets updated weekly.

Measurement Interval –

Monthly data collection beginning at the first day of the month.

Measurement Period – The full month, regardless of the number of days in the month.

Source of Measurement Data – Data will be collected from reports provided by the Contractor. It may also be randomly collected from ITSM+ or other appropriate Government systems.

Method of Surveillance – Periodic inspection and random sampling. The Government will obtain data from delivered reports monthly. It may also randomly collect reports from ITSM+ or other appropriate Government systems.

Timing of Measurement – Measurements are taken within five (5) days of the end of the Measurement Period.

Reporting Frequency: - Reporting to commence upon Contract Award and continue throughout the Period of Performance.

Monthly reporting of SLA and / or QASP requirements.

Exceptional = 100% Very Good > 95% Satisfactory = 95% Marginal = 90-94% Unsatisfactory < 90%

Contractor’s Executive Leadership Responsiveness to Performance Deficiencies

Target Criteria:

100% of deficiencies elevated to Contractor’s Executive Leadership level are not repeated and immediate corrective action is taken.

Minimum Acceptable:

95% of deficiencies elevated to Contractor’s Executive Leadership level are not repeated and immediate corrective action is taken.

Calculation:

Number of escalations to Contractor’s Executive Leadership where corrective action is taken, and issues are not repeated.

Measurement Interval –

Random. Issues are not escalated often enough to measure periodically.

Measurement Period – Random. Issues are not escalated often enough to measure periodically.

Source of Measurement Data – Data will be collected from reports provided by the Contractor. It may also be randomly collected from ITSM+ or other appropriate Government systems.

Method of Surveillance – Random Sampling. The Government will obtain data from delivered reports. It may also collect reports from ITSM+ or other appropriate Government systems.

Timing of Measurement – Random. Issues are not escalated often enough to measure periodically.

Reporting Frequency: - Random. Issues are not escalated often enough to measure periodically.

Exceptional = 100% Very Good > 95% Satisfactory = 95%

Contract Deliverables Reports (All PWS Tasks, except the specific metrics listed in this table) Target Criteria > 95% on time, complete, and accurate

Minimum Acceptable > 90% on time, complete, and accurate

Any required revisions are:

· Resolved within 1 week

· Accepted by KO/COR

Monthly

Measurement Period The Measurement Period begins at 00:01 on the first day of the month and ends at 24:00 on the last day of the month.

Source of Measurement Data Audit of deliverables or manual review.

Method of Surveillance Periodic Inspection.

Timing of Measurement Measurements are taken within five (5) Calendar days of the end of the Measurement Period.

Reporting Frequency Reporting to commence upon Contract Award and continue throughout the Period of Performance.

Exceptional = 100% Very Good > 95% Satisfactory = 95%

Appendix 2 – Performance Assessment Report

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

4. QUALITY ASSURANCE PERSONNEL (COR) SIGNATURE AND DATE

I. PERFORMANCE

7. |_| DEFICIENCY (CHECK ALL BOXES THAT APPLY)

|_| NEW

|_| REPEAT

|_| NO DEFICIENCY NOTED

8. SERVICES SUMMARY or PWS PARAGRAPH ITEM REVIEWED

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE |_| CONCUR |_| NON-CONCUR

12. CORRECTIVE ACTION ESTIMATED COMPLETION DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON FOR NON-CONCURRENCE OF COR CITED DEFICIENCY

III. ACTION CORRECTED

14. |_| CONCUR |_| NON-CONCUR COR SIGNATURE AND DATE

15. COR REMARKS (REQUIRED)

6. CONTRACTOR REPRESENTATIVE REMARKS

Appendix 3 – Customer Complaint Record

CUSTOMER COMPLAINT RECORD

DATE/TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION

BUILDING NUMBER

INDIVIDUAL

PHONE NUMBER

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDATED BY

Appendix 4 – Corrective Action Report

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6. CONTROL NUMBER

7. DEFICIENCY |_| MAJOR |_| MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

|_| ACCEPTED |_| REJECTED

12. CLOSE DATE

Controlled By: DISA/PS8222 CUI Category: Source Selection Distribution/Dissemination Control: FEDCON POC: disa.scott.ditco.mbx.ps822-satcom-contracting-branch@mail.mil