Attachment_3_QASP.pdf

FA4600-15-R-0022

Attachment 3

QUALITY ASSURANCE SURVEILLANCE PLAN

GLOBAL LIGHTNING DATA

21 September 2016

MULTI-FUNCTIONAL TEAM MEMBERS:

Date:

Functional Commander/Director

Date:

Contracting Officer Representative

Date:

Contract Administrator

Date:

Quality Assurance Program Coordinator

Date:

Contracting Officer

Date:

Contractor

TABLE OF CONTENTS

Subject Paragraph Number Page Number

1. Overview 1.0. 3

2. Description of Services 2.0. 3

3. Contract Quality Requirements 3.0. 3

4. Performance Assessment Approach 4.0. 3

5. Performance Assessment Documentation 5.0. 6

6. Duties and Responsibilities 6.0. 7

7. Performance Objectives 7.0. 9

8. Method of Performance Assessment 8.0. 11

9. Performance Metrics 9.0. 11

Acronyms:

• A&AS Administrative and Assistance Services

• CA Contract Administrator

• CAR Corrective Action Reports

• CCR Customer Complaint Records

• CID Contractor Identified Discrepancy

• CO Contracting Officer

• COR Contracting Officer’s Representative

• COS Certificate of Service

• CPARS Contractor Performance Assessment Reporting System

• ECD Estimated Completion Date

• GID Government Identified Discrepancy

• HMMP Hazardous Materials Management Process

• IA Issuing Authority

• LOA Letter of Acceptance

• MOASP Management of Oversight of the Acquisition of Services Process

• PAF Performance Assessment Folder

• PAR Performance Assessment Report

• PM Program Manager

• PO Performance Objectives

• PWS Performance Work Statement

• QAPC Quality Assurance Program Coordinator

• QASP Quality Assurance Surveillance Plan

• QCP Quality Control Program

• SS Services Summary

1.0. OVERVIEW

1.1. Purpose. The purpose of this Quality Assurance Surveillance Plan (QASP) is to identify the methods and procedures the Government will use to insure it receives the services under contract as identified in the Performance Work Statement (PWS). This plan will focus on the level of performance required by the PWS, not the methodology or process.

1.2. Intent. The intent of this QASP is to rely on the Contractor’s quality control, thus changing the Government’s role from “oversight” to “insight.” Performance Objectives (POs) will be periodically monitored and the quality program evaluated but the Contractor is left as free as possible to develop the most efficient methodology to meet or exceed the required thresholds of service.

1.3. Compliance with AFI 63-138, AFFARS MP5301.602-2(d). The QASP is an evolving document, the development of which begins with acquisition planning, and finalized as the acquisition progresses. The members of the multi-functional team sign the QASP. This plan identifies:

1.3.1. Objective(s) for having the service provided, i.e., to provide global lightning data for Air Force Weather. Refer to Paragraph 2.3.

1.3.2. Results the multi-functional team is striving to achieve in managing the acquisition, e.g., cost savings, efficiencies, and improved customer service. Refer to Paragraph 6.6.

1.3.3. A distribution of the roles and responsibilities among the multi-functional team members. Refer to Section 6.0.

1.3.4. The strategy, methods, and tools the multi-functional team will use to assess the Contractor’s performance against the performance thresholds, measurements, metrics, and incentives identified in the contract. Performance thresholds must be measurable in terms of quality and timeliness of performance.

Refer to Section 7.0.

1.3.5. The management approach, methods, and tools the multi-functional team will routinely use to validate the objectives and goals identified as part of the QASP. Refer to Section 4.0.

2.0. DESCRIPTION OF SERVICES

2.1. Scope of Work/Objective. The objective is to purchase global lightning data which provides Air Force weather forecasters with near-real time lightning data used for situational awareness and prediction of weather events.

2.2. Contract. The Global Lightning Data Contract is a Firm Fixed Priced (FFP) type contract. The Inspection of Services FAR Clause 52.246-4 applies.

3.0. CONTRACT QUALITY REQUIREMENTS

3.1. Quality Control Program (QCP). The Contractor shall develop and maintain a QCP to ensure services are performed in accordance with commonly accepted commercial practices. At a minimum, the Contractor shall develop quality control procedures addressing the areas identified in Section 7.0., Performance Objectives. The Contractor’s QCP shall be submitted to the Government within 30 days of contract award for the Quality Assurance Program Coordinator and Contracting Officer’s review, feedback if necessary, and Contracting Officer’s approval.

4.0. PERFORMANCE ASSESSMENT APPROACH

4.1. Purpose. This section details the method(s) used to verify Contractor compliance with PWS requirements. The key elements of this process are the Contractor’s QCP and Government identified Performance Objectives. The Performance Objectives from Paragraph 7.0. of this QASP dictate the minimum inspection requirements Contracting Officer’s Representative (COR) must accomplish on a periodic basis.

This QASP provides the procedures on how to conduct the evaluations for Performance Objectives and how to document them in a correct and effective manner.

4.2. Performance Assessment Approach. To facilitate the performance assessment of the Contractor’s QCP, the COR will verify Contractor compliance with the designated Performance Objectives. The intent of the performance assessment approach is to gain confidence in the Contractor’s ability to provide satisfactory services and then adjusting the level of performance assessment to a point that maintains confidence. This performance assessment approach is subject to change based on the Contractor’s performance.

4.3. Annual Review. The COR must review the Performance Objectives, assess their applicability and recommend the addition or subtraction of Performance Objectives as conditions warrant. This review must be accomplished annually at a minimum.

4.4. Special Audits. The COR must be alert to conditions that would warrant a special quality audit. Any time the COR observe that a functional area is out of acceptable tolerance or that the technical expertise is not available, a request for a functional area quality audit should be addressed to the Contracting Officer (CO).

4.5. Electronic File. The COR shall maintain an electronic file in the CORT-Tool as prescribed in accordance with DFARS PGI 201.602-2(d)(vi).

4.6. Deficiency Identification. Effective contract performance assessment relies on identification of unacceptable performance, either Contractor Identified Discrepancies (CIDs) and Government Identified Discrepancies (GIDs).

4.6.1. Contractor Identified Discrepancy (CID). CIDs are generated when contract employees detect discrepancies during performance of routine daily services or through the Contractor’s QCP. When CIDs are reviewed by the applicable COR, that person will take the following actions:

4.6.1.1. Review corrective actions taken by the Contractor to resolve a discrepancy.

4.6.1.2. Determine whether the discrepancy has been corrected. If the discrepancy has been resolved, review the corrective action taken and any applicable procedures the Contractor has established to preclude recurrence of the problem.

4.6.1.3. If a CID has not been resolved and the Contractor has established an Estimated Completion Date (ECD) for resolution of the discrepancy, the COR will monitor the Contractor’s efforts in resolving the discrepancy. If the COR determines the estimated completion date for resolution of the discrepancy is excessive based on the nature of the discrepancy, the CO will be notified. The CO can then evaluate the recommendation to determine the appropriate course of action.

4.6.1.4. If a CID has not been resolved and the COR determines that the Contractor has not established an ECD for resolution of the discrepancy, or has failed to perform a follow-up action for resolution of a discrepancy, the COR will issue a CAR. Refer to Paragraph 5.1. The Contractor will be given a suspense date to provide corrective action to the Government’s findings and describe the Contractor’s plan to preclude repeat deficiencies.

4.7. Government Identified Discrepancy (GID). GIDs are generated as a result of the COR’s inspection of Contractor performance that identifies noncompliance with contractual requirements. The COR will record a

GID on a CAR. Refer to Paragraph 5.1. Before a CAR is generated, the Contractor will be given the opportunity to show whether the discrepancy has already been identified. If documentation exists which substantiates that the Contractor’s quality program has identified the discrepancy and timely corrective action is being taken to address the discrepancy, a CAR will not be initiated.

4.8. Customer Complaints. When customer complaints are to be used as the method of performance assessment, the following procedures will be followed:

4.8.1. The COR will educate customers concerning the service(s) being provided by the Contractor.

4.8.2. The COR will provide a CCR which shall include clear and simple instructions to the customer for recording their comments. The form should allow for both positive and negative comments.

4.8.3. If immediate response to a complaint is needed the COR will provide the customer with instructions on how to proceed.

4.8.4. The COR will collect all completed CCR.

4.8.5. The COR will conduct an investigation to determine the validity of any negative comments received.

4.8.6. If a determination is made that a negative comment is not valid, the COR will inform the customer of the reason(s) why and carry on further correspondence if necessary.

4.8.7. For validated negative comments, the COR, in coordination with the CO, will notify the Contractor to allow the Contractor an opportunity for investigation and comment. The COR will refer to the procedures in Paragraph 5.0. to determine whether it is necessary to initiate a CAR.

4.8.8. The COR will also notify the customer of the corrective action taken by the Contractor, if applicable.

4.8.9. The COR, with assistance from the CO, will conduct trend analysis of validated negative comments.

5.0 PERFORMANCE ASSESSMENT DOCUMENTATION

5.1 Corrective Action Report (CAR). A CAR is used by the COR to document contract requirement findings and subsequent corrective actions to be taken. A computer-generated CAR may be used to provide the information on the CAR but must include a handwritten signature. See attached CAR and instructions for completion.

5.1.1. The COR identifying the discrepancy is responsible for completing the CAR utilizing the attached instructions. If there are other questions as to how the CAR is completed that individual should consult the CO.

5.1.2 Consult the assigned Contract Administrator to give them the opportunity to assist and advise how the CAR is to be worded. Forward a copy of the initiated CAR to the Contracting Officer for their signature in Block 9 as the Issuing Authority. The Contracting Officer will forward the CAR to the contractor requesting a written corrective action plan within 10 days.

5.1.3 The COR will elevate any CARs that cannot be resolved with the Contractor to the CO for resolution.

5.1.4 Categories of Findings. A finding is any Government-identified noncompliance with contract requirements which specifies that an activity or action did not take place or was not performed to the quality or timeliness standard required by the contract. A finding can be categorized as either a “Major Finding” or a “Minor Finding.” Findings will be documented on a CAR and must also be tracked in the activity log.

Major/Minor Findings are defined as follows:

5.1.4.1 Major Finding. A conclusion that identifies a condition having a significant adverse effect on the quality of the activity under review such as:

a. Failure to meet a Performance Threshold.

b. Significant mission degradation in any Contractor operated function.

c. A situation that is likely to result in hazardous or unsafe conditions for individuals (or any other Government resource) coming into contact with the work.

d. Failure to provide adequate corrective action to preclude recurrence of government identified findings.

e. Failure to provide corrective action to deficiencies identified by the Contractor within a prescribed suspense period.

f. Any failure to adhere to security regulations that results in a security incident.

5.1.4.2 Minor Finding. A departure from established standards having little bearing on service provided. However, if the same minor finding is repeatedly identified, it may be an indication that Major Finding “d.” above is occurring or has occurred because the Contractor quality system has not acted to prevent reoccurrences. In this case, the Minor Finding should be elevated to the level of a Major Finding and a new CAR initiated.

5.2 Performance Assessment Report (PAR). The COR must document performance assessment as it is accomplished quarterly. The PAR is an official government record of inspections of the performance objectives and other inspection items contained in this QASP. The PAR is attached to this QASP.

5.2.1 Completed PARs will be maintained in the COR PAF, and an electronic copy submitted to the CA and uploaded to the CORT Tool website (https://arc.army.mil/dodcor/) upon completion of each PAR. At the completion of the contract, the COR Supervisor will contact the CO for disposition instructions of the PARs. These reports may be maintained in hard-copy form as long as there is ample backup to preclude loss of data.

6.0 DUTIES AND RESPONSIBLILITIES/REPORTING REQUIREMENTS

6.1 COR Inputs to the FC/D. The COR is required to report detailed results of Performance Objective verifications to the FC/D on an as needed basis.

6.2 COR Supervisor.

6.2.1 Identifies mission essential services, to include developing the necessary documents in accordance with DoDI 3020.37, Continuation of Essential DoD Contractor Services During Crisis.

6.2.2 Assigns Primary and Alternate COR unless another Quality Assurance organization or method can satisfy performance assessment requirements (e.g., Program Management Office).

6.2.3 Reviews Contractor performance documentation prepared by COR on a regular basis to ensure performance is compatible with contract and mission objectives while supporting a higher level of Contractor performance, and fostering synergistic partnerships.

6.3 Contracting Officer Representative.

6.3.1 Designated to perform quality assessment functions and manage performance in accordance with the QASP. They serve as on-site technical managers assessing Contractor performance against performance thresholds indicated in the service summary.

https://arc.army.mil/dodcor/

6.3.2 Evaluate and document Contractor’s performance in accordance with the QASP.

6.3.3 Notify the CO of any significant performance deficiencies.

6.3.4 Maintain surveillance documentation.

6.3.5 Recommend improvements to the QASP and PWS throughout the life of the contract.

6.4 Contracting Officer/Contract Administrator (CO/CA).

6.4.1 The CO/CA are responsible for contract management and implementation of the QASP after contract award by the multi-functional team.

6.4.2 The CO/CA integrate the goals and objectives identified by the multi-functional team, and provide oversight, performance management, and performance measurement in accordance with the QASP.

6.4.3 Delegate authority for inspection and/or acceptance in accordance with terms of the contract and informs the Contractor of the names, duties, and limitations of authority for all quality assurance personnel assigned to the contract.

6.5 Quality Assurance Program Coordinator (QAPC). Mission Support Group or AFMC/AFSPC Center-level individual, normally from the contracting activity, selected to coordinate and manage the Performance Management Assessment Program (Quality Assurance Program).

7.0 PERFORMANCE OBJECTIVES

7.1 Performance Objectives (PO). The following is a list of the contractual PO to be validated by COR.

Each PO includes a government-determined performance threshold and the method of performance assessment. The PO are arranged according to the functional area they are associated with. Functional areas not listed were omitted because they did not have PO identified as critical. The PO in this paragraph represent the minimum items that must have a performance assessment accomplished on a periodic basis. Any performance assessment of PWS requirements not required by the QASP must still be documented in the PAR as described in Paragraph 4.2.

7.2 Services Summary (SS). The Contractor service requirements are summarized into PO that relate directly to mission essential items. The performance threshold briefly describes the minimum acceptable levels of service required for each requirement. These thresholds are critical to mission success. Contractor monthly payments may be adjusted based upon the performance thresholds established in the SS for any PO that the number of allowable defects is exceeded for a given month.

Table 7-2.1: Performance Objectives for Global Lightning Data Delivery and Vendor Support

PERFORMANCE

OBJECTIVE

PWS

PARAGRAPH

PERFORMANCE THRESHOLD

(Acceptable Quality Level)

METHOD OF

ASSESSMENT

Data Delivery Format

3.3 Streaming ACSII text or XML format. Customer

Complaint

Data Stream Content Elements

3.4 Stream elements specified in PWS

para 3.4.1

Customer Complaint

Data Stream Format

3.4 Data received in Stroke format Customer

Complaint

Data Encrypted 3.5

Data received encrypted Customer Complaint

3.6 Data delivered via the internet Customer

8.0 METHOD OF PERFORMANCE ASSESSMENT.

8.1 Surveillance Techniques. In an effort to minimize the performance management burden, simplified surveillance methods shall be used by the Government to evaluate contractor performance when appropriate.

The method of surveillance is customer complaint – Relies on the user of the service to identify deficiencies.

Complaints are then investigated and validated by the COR against the PWS.

9.0 PERFORMANCE METRICS. Used to track Contractor progress towards meeting stated PO. The multi-functional team in assessing Contractor performance validates that the performance metrics align with the performance-based work statement and overall mission support objectives.

Data Receipt Method

Data Delivery 3.7

Delivered in four data feeds Customer

Data Availability 3.8 24 hours per day, 7 days per week per data requirements in Tables 2 to 4 in PWS tables 2-4 on a 24/7 basis.

Service Issue Notification

3.8 Notified when data feed compromised.

Network Availability

3.9 98% uptime Customer

Complaint

Data Availability by Zone Except Zone 15

3.9 98% uptime Customer

Help Desk 3.10

Acknowledge calls 24/7/365, initial response within ten (10) hours after receiving service call.

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES / NAME OF CONTRACT

4. FUNCTIONAL AREA

5. SUSPENSE DATE

(FOR CONTRACTOR)

6. CONTROL NUMBER

7. DEFICIENCY MAJOR MINOR (AS DEFINED IN THE QUALITY ASSURANCE SURVIELLANCE PLAN (QASP)

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. CONTRACTING OFFICER’s REPRESENTATIVE (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY (COR SUPERVISOR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

(Attach the Contractor’s Corrective Action Plan if Provided. State N/A if an Action Plan was not required.)

11. COR DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE

Corrective Action Report (CAR) – 55 CONS – JAN 2013

CORRECTIVE ACTION REPORT (CAR)

Continuation Sheet

(Number to correspond with applicable Item Number on reverse)

Corrective Action Report (CAR) – 55 CONS – JAN 2013

Corrective Action Report (CAR) Instructions

Block 1. Enter Contractor Name.

Block 2. Enter Contract Number.

Block 3. Enter Contract Name or Type of Services.

Block 4. Enter Functional Area of the Contract.

Block 5. Enter the assigned suspense date given the contractor to provide a response to the CAR. A date must be entered for a Major CAR. A date is optional at the discretion of the Contracting Officer’s Rep (COR) initiating the CAR if the finding is Minor.

Block 6. All CARs will be tracked with a Control Number. The COR Supervisor is the Issuing Authority for his/her contract Quality Assurance Evaluators (CORs) generating CARs. By providing the control number to the COR originating the CAR, the Issuing Authority demonstrates concurrence with the finding. The control number will be made up of the first two letters of the name of the site or contract, the last two digits of the calendar year, and a three digit number starting with 001 and progressing upward throughout the calendar year (e.g., WP13-001).

Block 7.

1. Check the block that indicates whether the identified deficiency is assigned as a Major or Minor finding. Refer to the Quality Assurance Performance Plan (QASP) for the definitions of Major and Minor findings and associated explanations.

2. Finding: Clearly state the details of the finding followed by a reference to the stated contractual requirement. The reference must state the portion of the contract, part, section, paragraph and subparagraph and must make a complete brief quotation of the contract reference.

3. State the impact that the finding has or could have on the accomplishment of the mission that the contract provides.

4. When determining how long to give the contractor to respond to the identified finding, the normal is 10 working days.

The suspense date should reflect this unless the finding requires a greater amount of time to come to solution. Major findings require a contractor response. Minor findings may require a contractor response at the discretion of the COR initiating the CAR and/or the Issuing Authority.

Blocks 8 and 9. COR’s initiating a CAR must sign in Block 8 and the Issuing Authority signs in Block 9. Each annotates the date at the time of signature.

Block 10. Upon review of the contractor’s reply, the originator of the CAR will enter comments regarding acceptance or rejection of the contractor’s response. This block may also contain any comments regarding follow-on inspections conducted or needed at a later date to validate that the finding has not reoccurred.

Block 11. The COR selects Accept or Reject after reviewing the contractor’s response and document as needed.

Block 12. The COR enters a close date after advising the Issuing Authority that the contractor’s response is acceptable and the Issuing Authority concurs. The Issuing Authority will then forward the closed CAR to the contractor and report accordingly in the monthly Certificate Of Service (COS) or other applicable contracting officer services report. This routing remains the same if the contractor response is unacceptable and the CAR remains open until an acceptable response is received except that the Issuing Authority forwards the open CAR back to the contractor for additional information.

Continuation Sheet. The Continuation Sheet is to be used to expand on any information that will not fit in the applicable numbered block.

CUSTOMER COMPLAINT RECORD

DATE/TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION

BUILDING NUMBER

INDIVIDUAL

PHONE NUMBER

NATURE OF COMPLAINT

CONTRACT REFERENCE (Input By The COR) (eg Paragraph From Performance Work Statement)

VALIDATION by the COR VALID COMPLAINT NOT VALID

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT on the Corrective Action Report (CAR)

ACTION TAKEN BY CONTRACTOR (Filled out by the COR)

RECEIVED/VALIDATED BY (Signature of COR)

Customer Complaint Record - JANUARY 2013 (55 CONS)

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACT/TASK ORDER NUMBER 2. CONTRACTOR 3. TYPE OF SERVICES

4. CONTRACTING OFFICER’s REPRESENTATIVE (COR) SIGNATURE AND DATE

5. COR PHONE 6. SUSPENSE DATE

I. PERFORMANCE

7. DEFICIENCY (CHECK ALL BOXES THAT APPLY)

NEW

REPEAT

NO DEFICIENCY NOTED

8. SERVICES SUMMARY or PWS PARAGRAPH ITEM REVIEWED

9. BRIEF DESCRIPTION OF DEFICIENCY (IF DEFICIENCY BOX WAS

CHECKED)

10. DETAILED PERFORMANCE ASSESSMENT

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE CONCUR NON-CONCUR

12. CORRECTIVE ACTION ESTIMATED COMPLETION DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON FOR NON-

CONCURRENCE OF COR CITED DEFICIENCY

III. ACTION CORRECTED

14. CONCUR NON-CONCUR COR SIGNATURE AND DATE

15. COR REMARKS (REQUIRED)

16. CONTRACTOR REPRESENTATIVE REMARKS

Performance Assessment Report (PAR) -- 55 CONS -- JAN 2013

PERFORMANCE ASSESSMENT REPORT

Continuation Sheet

(Number to correspond with applicable Item Number on reverse)

Performance Assessment Report (PAR) – 55 CONS – JAN 2013

Performance Assessment Report (PAR) Instructions

Block 1. Contracting Officer’s Representative (COR) enters contract or task order number.

Block 2. COR enters contractor name.

Block 3. COR enters type of services.

Block 4. COR signs and dates.

Block 5. COR enters telephone number.

Block 6. COR assigns suspense date for Contractor Representative validation if a deficiency box was checked in Block 7.

Part I. Contractor Performance

Block 7. COR checks all boxes that apply.

Block 8. COR enters Services Summary or Statement of Work (SOW) paragraph item reviewed.

Block 9. COR enters brief description of deficiency and tracking number if the deficiency box was checked in Block 7.

Detailed description will be on Corrective Action Report (CAR).

Block 10. COR enters a detailed performance assessment if the deficiency box in Block 7 was not checked.

Parts II and III will be used only if a deficiency was documented in Part I.

Part II. Contractor Validation

Block 11. Contractor Representative checks one box in response to COR cited deficiency. If Concurrence is indicated, Contractor Representative continues to Block 12. If Non-Concurrence is indicated, continue to Block 13 and state reason(s) for Non-Concurrence with COR cited deficiency and return PAR to COR.

Block 12. Contractor Representative enters Estimated Completion Date of Corrective Action and continues to Block 13.

Block 13. Contractor Representative details Corrective Action and the Action Taken to Prevent Recurrence or Reason for Non-Concurrence of COR cited deficiency.

Part III. Deficiency Corrected

Block 14. COR checks appropriate block, signs, and dates.

Block 15. (Required Entry) COR remarks regarding the Corrective Action and the Action Taken to Prevent Recurrence. If Non-Concur was checked, this block should be very detailed to explain why.

Block 16. Contractor Representative enters remarks if further discussion is needed about COR concurrence with Contractor’s Corrective Action and Action Taken to Prevent Recurrence.

Continuation Sheet. The Continuation Sheet is to be used to expand on any information that will not fit in the applicable numbered block.