Attch_5 _QASP.pdf

FOR OFFICIAL USE ONLY

SOURCE SELECTION SENSITIVE

Quality Assurance Surveillance Plan (QASP)

For

Integrated Logistics Support / Contractor Logistics Support

(ILS/ CLS)

Office of Security Capabilities Transportation Security Administration

April 18, 2013

Transportation Security Agency Quality Assurance Surveillance Plan

___________________________________________________________________________1-2

TABLE OF CONTENTS

1. INTRODUCTION

1.1 OVERVIEW

1.2 PURPOSE

1.3 QASP RELATION TO OVERALL PERFORMANCE MANAGEMENT APPROACH

1.4 QASP RELATION TO THE QSP

1.5 QASP REVISIONS

2. ROLES AND RESPONSIBILITIES

2.1 CONTRACTOR RESPONSIBILITY

2.2 GOVERNMENT RESPONSIBILITY

2.2.1 Contracting Officer (CO)

2.2.2 Contracting Officer’s Representative (COR)

2.2.3 Quality Assurance Evaluators (QAE) ......................................................... Error! Bookmark not defined.

3. PERFORMANCE DESCRIPTION

3.1 PERFORMANCE STANDARDS AND ACCEPTABLE QUALITY LIMITS (AQLS)

3.2 RATINGS

3.3 DOCUMENTING PERFORMANCE

3.3.1 Acceptable Performance

3.3.2 Unacceptable performance

3.4 REMEDIAL ACTIONS

4. PERFORMING QUALITY ASSURANCE

4.1 SURVEILLANCE METHODS

4.1.1 Direct Observation

4.1.2 100% Inspection

4.1.3 Periodic Inspection

4.1.4 Random Sampling

4.2 LEVELS OF INSPECTION

5. DOCUMENTATION AND REPORTS

5.1 DOCUMENTATION REQUIREMENTS

5.2 REPORTS

5.2.1 Quality Assurance Monitoring Report & Log

5.2.2 QA Report

APPENDIX A: PERFORMANCE REQUIREMENTS SUMMARY (PRS)

APPENDIX B: QUALITY ASSURANCE MONITORING REPORT FORM

APPENDIX C: QUALITY ASSURANCE MONITORING LOG FORM

APPENDIX D: CONTRACT DISCREPANCY REPORT

APPENDIX E: ACRONYMS

___________________________________________________________________________1-3

1. INTRODUCTION

1.1 Overview

This Quality Assurance Surveillance Plan (QASP) identifies the technical performance requirements for ILS CLS services and the surveillance and evaluation techniques that will be used to assure that TSA is receiving the standard of service detailed in the PWS.

TSA retains the right to change the surveillance methods and QA procedures, and to increase or decrease the degree of surveillance efforts at any time necessary to assure contract compliance. However, the Contractor is responsible for assuring only conforming material and services are offered to the U.S.

Government for acceptance. Any discovered contractual nonconformance not specifically identified for surveillance/measure within this document requires responsive Contractor corrective action.

1.2 Purpose

The QASP is established to evaluate Contractor technical performance for the purpose of assuring the Contractor meets the performance standards of this contract.

The QASP is intended to accomplish the following:

• Identify the TSA Office of Security Capabilities (OSC) procedures and guidelines used to evaluate the performance of the Contractor.

• Define the role and responsibilities of participating TSA officials.

• Define the key service levels that will be measured.

• Describe the rating criteria and evaluation methods employed by TSA in accessing the

Contractors performance.

• Identify the forms, reports and resources used to document TSA’s monitoring efforts and results.

1.3 QASP Relation to Overall Performance Management Approach

The documented results of the QA surveillance will be used by TSA to monitor, manage, and take appropriate action on the Contractor’s performance against specified contract requirements. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management.

The data generated by the QA surveillance provides information that indicates whether or not expected outcomes for required services are being achieved adequately by the contractor.

1.4 QASP Relation to the QMP

The Contractor’s quality management system, as documented in their quality management plan (QMP), is a required element of the contract that sets forth the process and procedures the Contractor will utilize to ensure the quality and timeliness of services delivered as defined in the PWS. The QASP represents the way in which TSA will evaluate the Contractor’s performance. The Contractor’s QMP and TSA’s QASP should be complementary programs that ensure the Contractor’s performance meets the standards defined in the PWS.

1.5 QASP Revisions

The QASP is a tool for use in TSA administration of the contract, and remains subject to revision at any time by TSA throughout the contract performance period. Revisions to this surveillance plan are the responsibility of the Contracting Officer’s Representative (COR). Any anticipated changes to this document will be announced to the contractor prior to implementing any revision(s).

Transportation Security Agency Draft Quality Assurance Surveillance Plan

___________________________________________________________________________1-4

FOR OFFICIAL USE ONLY

SOURCE SELECTION SENSITIVE

2. ROLES AND RESPONSIBILITIES

The purpose of QA is to ensure that the customers are satisfied with the materials and services received from the Contractor, and to ensure that the Contractor is meeting its obligation to TSA. The roles and responsibilities of the stakeholders involved in QA are described below.

2.1 Contractor Responsibility

The Contractor is responsible for delivering all products and services in accordance with the contract.

The Contractor is responsible for implementing a quality program for producing, maintaining, and providing for audit, quality control records and reports, and all records associated with the investigation and resolution of problems in instances when contractor fails to comply with contract requirements.

2.2 Government Responsibility

The Contracting Officer (CO) and the Contracting Officer’s Representative (COR) are the key Government personnel who will be responsible for QA.

2.2.1 Contracting Officer (CO)

The CO has the authority to administer the contract, ensures compliance with the terms of the contract, and safeguards the interests of TSA in the contractual relationship. The CO may delegate many of the day-to-day contract administration duties to the COR. However, certain contractual actions such as negotiation and issuance of contract modifications, resolution of Contractor claims and disputes, issuance of cure notices, issuance of show-cause letters (following a cure notice, requesting facts bearing on the case), termination of the contract, and contract close-out functions are retained by the CO.

The CO ensures the Contractor receives impartial, fair, and equitable treatment under the contract. The CO is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

All communication regarding questions or issues related to QA and inspection will be directed to the CO or the COR. Any changes that the Contractor deems may affect contract, price, terms, or conditions may only be referred to the CO for action. The CO shall approve any revision to the QASP processes or standards.

In accordance with Federal Acquisition Regulation (FAR) 42.302(a) (57), the CO also delegates specific surveillance tasks and authorities by formal letter to the COR. Administrative actions such as invoice approval and issuance of Contract Discrepancy Reports may be delegated by the CO to the COR.

2.2.2 Contracting Officer’s Representative (COR)

The COR is the individual designated in writing by the CO to act as a liaison between TSA and the Contractor on all technical issues pertinent to the daily operation of the contract. COR limitations are contained in the written letter of designation and Section B.3.4.2 of the contract. The COR is not authorized to make any commitments or otherwise obligate the Government or authorize any changes which affect the contract’s price, terms or conditions.

The COR is responsible for the technical aspects of the contract and is therefore the Contractor’s initial point-of-contact with TSA to ensure that the QA function is properly executed.

For tasks and/or subtasks which include performance incentives, the COR will provide recommendations to the CO for action. If modifications to the contract are necessary, the COR will assist the CO in preparing and negotiating the modifications. If technical performance is incomplete or deficient, the COR will notify the Contractor and request corrective actions are taken. The CO will make final determination on performance issues that affect contract cost. The COR will document deficient performance in the file for historical Contract Past Performance.

___________________________________________________________________________1-5

FOR OFFICIAL USE ONLY

SOURCE SELECTION SENSITIVE

The COR performs contract surveillance and may document results using the forms provided in Appendix B. The form documents the specific tasks to be checked, and whether the inspection results in a Contractor rating of excellent, compliant or noncompliant performance.

The COR will be knowledgeable about the products and/or services provided by the Contractor, able to identify discrepancies in product and/or services, and is responsible for documenting and reporting all inspection observations.

3. PERFORMANCE DESCRIPTION

ILS/ CLS services are provided by the Contractor in accordance with the PWS and the specified performance/ service level requirements.

Contractor performance will be monitored through various surveillance methods described in Section 4.1.

This plan describes the methodology utilized to make both quantitative and qualitative evaluation of Contractor performance under the contract.

3.1 Performance Standards and Acceptable Quality Limits (AQLs) For selected activities in the PWS, the Performance Requirements Summary (PRS) in Appendix A provides the PWS performance standards and AQLs. A performance standard is the expected level of Contractor performance, and the AQL defines the level of performance that is satisfactory.

The contract requires all work be performed as specified. Any inaccuracies, omissions or failures to meet contractual requirements in services are referred to as discrepancies. The Contractor will be held responsible for all identified discrepancies, and TSA may require the Contractor to re-perform or correct the work at no additional cost to TSA.

3.2 Ratings

Metrics and methods are designed to determine if performance exceeds, meets, or does not meet a given performance standard. A rating scale shall be used to determine an excellent, compliant, or non-compliant outcome as described below:

EXCELLENT:

Performance significantly exceeds contract requirements to the Government’s benefit.

COMPLIANT: Performance meets contractual requirements.

NON-COMPLIANT: Performance does not meet contractual requirements.

3.3 Documenting Performance

3.3.1 Acceptable Performance

The Government will document acceptable performance. Any report may become a part of the documentation supporting contract past performance.

3.3.2 Unacceptable performance

When unacceptable performance occurs, the COR will inform the Contractor, with notification to the CO.

This will normally be in writing unless circumstances necessitate verbal communication. In any case the COR will document the unacceptable performance in the COR file.

___________________________________________________________________________1-6

FOR OFFICIAL USE ONLY

SOURCE SELECTION SENSITIVE

When the CO or COR determine formal written communication is required, the COR shall prepare a Contract Discrepancy Report (CDR), and present it to the contractor's Program Manager. Appendix D includes a CDR template. The Contractor shall acknowledge receipt of the CDR in writing.

The CDR will specify if the Contractor is required to prepare a corrective action plan and when the corrective action plan must be submitted to the COR. If required, corrective action plan shall document how the Contractor will correct the unacceptable performance and avoid a recurrence. The Government will review the Contractor's corrective action plan to determine acceptability.

Any CDRs may become a part of the supporting documentation for contract payment deductions or other actions deemed necessary by the Contracting Officer.

3.4 Remedial Actions

The FAR allows for penalties in the event that the Contractor fails to perform the required services.

Penalties are defined as those actions taken under the direction of the CO against the Contractor within the general provisions of the contract for noncompliance to the PWS and PRS.

In accordance with FAR 46.102 and FAR 46.105, the Government will not accept nonconforming products and/or services. The Contractor is required to correct products and/or services that fail to meet contract requirements, at no increase in contract price. If the Contractor fails to proceed with reasonable promptness to perform the required correction, the Government may take actions to collect consideration for work not accomplished.

4. PERFORMING QUALITY ASSURANCE

The COR performs contract surveillance using various methods to either confirm the Contractor’s successful achievement of performance requirements, or highlight areas where discrepancies exist and improvements are necessary.

The COR is responsible for developing a monthly schedule of surveillance activities based on QASP requirements. The monthly schedule should be completed not later than five (5) business days before the beginning of the period it covers. The COR must retain a copy of the schedule for the documentation files.

4.1 Surveillance Methods

The following performance monitoring methods can be applied individually or in combination. The method or combination of methods used when conducting an inspection should be noted when describing the surveillance methods on the QA Monitoring Form found in Appendix B. Surveillance methods should be chosen and implemented with the goal of minimizing intrusive disruption.

Management Information Systems (MIS) reports can assist 100% inspection, random sampling or periodic inspection, and may be used to evaluate outputs through the use of management information reports. When using Contractor-generated reports, this method is best for general surveillance, and may need to be supplemented by periodic inspections. The QAE may accept Contractor-generated reports at face value, but use other methods to investigate problem areas.

4.1.1 Direct Observation

Direct observation of Contractor performance can be performed periodically or through 100% inspection.

Direct Observations may be scheduled or unscheduled. Scheduled Observations are previously scheduled

___________________________________________________________________________1-7

FOR OFFICIAL USE ONLY

SOURCE SELECTION SENSITIVE

and coordinated with the Contractor. Unscheduled Observations may be conducted at random by TSA personnel.

4.1.2 100% Inspection The 100% inspection method requires complete inspection of a contract requirement and will be used for requirements that are especially critical or where there is some reason for suspecting that the performance standard is not being met (and therefore, should be more closely monitored).

4.1.3 Periodic Inspection

Periodic inspection, sometimes called “planned sampling,” provides a systematic way of looking at service outputs and forming conclusions about the Contractor’s level of performance in accordance with a planned schedule of surveillance. Evaluation by periodic inspection is designed to inspect some part but not all of the products and services being monitored.

Periodic inspection is generally used in two ways. First, it can provide a one-time subjective evaluation of Contractor performance. Second, it can be used to detect a change in the Contractor’s level of performance (i.e., trend analysis). This method requires that the sample selection criteria be well documented and consistently applied from period to period, and that there are no other intervening factors.

With this type of evaluation, the COR is able to direct efforts to those areas where inspections are most needed, and the Contractor knows that those areas are more likely to be monitored than others. Periodic inspection provides a means to make a comparison between observed and reported performance, with the results being an indicator of the Contractor’s overall level of performance.

The levels of evaluation appropriate for periodic inspection are judgmental. In order to perform trend analysis from periodic inspection, criteria for sample selection should be applied consistently from period to period. Observed discrepancies monitored by periodic inspection will be totaled and assessed at the end of each evaluation period. For each requirement, the findings will be compared to the performance standard and AQL.

4.1.4 Random Sampling

Random Sampling is the most appropriate method for recurring tasks. This QA method is designed to evaluate Contractor performance by randomly selecting and inspecting a statistically significant sample.

Random sampling is recommended for large quantity repetitive activities with objective and measurable quality attributes. The random sampling procedures consider the AQL, the level of the evaluation effort, and the population size. There are two ways of applying random sampling for QA surveillance. The first is used only for performance evaluation, and allows deductions to be taken only for observed nonconformities; the second is random sampling for performance evaluation and deduction projection, which allows deductions against the whole population based on the inspection of the sample.

Random Sampling is based solely on a statistical analysis whereby a conclusion is drawn about a population based on a randomly selected sample of that population. For the conclusion to be valid, the sample selected must be representative of the population. A truly representative sample can be achieved by ensuring that the sample is selected randomly and the size of the sample is sufficient.

4.2 Levels of Inspection

Three levels of inspection “Normal,” “Reduced” and “Tightened”, are utilized for the random sampling surveillance method. It is expected that during the initial period following contract award, all operational procedures and QA measures will be tested and implemented. Normal inspection will be used upon the initialization of the contract, unless otherwise directed by the COR. As the performance period

___________________________________________________________________________1-8

FOR OFFICIAL USE ONLY

SOURCE SELECTION SENSITIVE

progresses, the levels of surveillance inspection may be altered based on COR performance ratings. The COR will notify the CO of proposed changes to the inspection level.

When normal inspection is in effect, tightened inspection will be instituted when ten percent (10%) of randomly sampled items are Unsatisfactory on original inspection (ignoring resubmissions). When tightened inspection is in effect, normal inspection may be reinstituted at the discretion of the COR when ninety-eight percent (98%) or randomly sampled items are rated Satisfactory on original inspection.

When normal inspection is in effect, reduced inspection may be instituted when one hundred percent (100%) of randomly sampled items are rated Satisfactory on original inspection for three consecutive rating periods.

5. DOCUMENTATION AND REPORTS

5.1 Documentation Requirements

Accurate and complete documentation is necessary for QASP administration. The COR will, in addition to providing documentation to the CO, maintain a complete QA file. The file will contain copies of all reports, evaluations, recommendations, surveillance checklists, and any actions related to the quality assurance surveillance function. All such records will be retained for the life of the contract and forwarded to the CO upon completion.

5.2 Reports

5.2.1 Quality Assurance Monitoring Report & Log

This report is used to document QAE surveillance as it is conducted (Appendices B & C).

5.2.2 QA Report

At the end of the evaluation period or as required by the CO, the COR will prepare a written report for the CO summarizing the overall results of the quality assurance monitoring of the Contractor’s performance.

This report, consisting of a summary, the Contractor’s submitted reports, and the completed Quality Assurance Monitoring Forms, will become part of the QA documentation.

___________________________________________________________________________1-9

SOURCE SELECTION SENSITIVE

APPENDIX A: PERFORMANCE REQUIREMENTS SUMMARY (PRS)

The COR will monitor Contractor performance using the methods in Section 4, together with the table below and the contract PWS. The PRS includes performance standards, AQLs, surveillance methods, and the impact to the contract for performance that does not meet standards. The Contractor is responsible for providing all products and/or services as required in the Contract. The following PRS summarizes the portions of the contracts on which TSA QA surveillance efforts are focused. At the direction of the COR, TSA reserves the right to alter the PRS at anytime during the contract to ensure the standard of services required by the contract is received.

REQUIREMENT PERFORMANCE

STANDARD AQL

SURVEILLANCE

METHOD

INCENTIVES (+/-)

System Operational Availability

(Ao)

Ao for each technology at each airport meets AQL, measured monthly

98%

100% Inspection of maintenance tickets submitted on CDRL

A004

10% withheld from monthly invoice for any technology at an airport not meeting AQL

Parts availability time (M4/M5)

For total monthly critical maintenance actions requiring parts (M4 & M5 > 0), the average time in hours for the sum of M4 & M5 is less than or equal to AQL

20 hours

100% Inspection of maintenance tickets requiring parts submitted on CDRL

A004

• Measured quarterly for all maintenance actions requiring parts over previous 3 months

• If average time for the sum of M4 & M5< 17.5, the Contractor earns 10% of annual incentive pool.

• If average time for the sum of M4 & M5< 15, the Contractor earns 25% of annual incentive pool.

Technician response time (M2)

For total monthly critical maintenance actions, the average time in hours for M2 is less than or equal to AQL

CAT X airports = 3 hours

All other airport categories =

5 hours

100% Inspection of maintenance tickets requiring dispatch submitted on CDRL A004

CO may seek remedy in accordance with the FAR for missed performance

Depot repair turnaround time (M11)

Monthly, the average time in calendar hours for M11 is less than or equal to AQL

360 calendar hours

100% Inspection of maintenance tickets requiring depot maintenance submitted on CDRL A004

• Measured quarterly for all maintenance actions requiring depot repair over previous 3 months

• If M11 < 300 hours, the Contractor earns 10% of annual incentive pool.

• If M11 < 240 hours, the Contractor earns 25% of annual incentive pool.

Radiation survey reports are provided to the local TSA

POC

Radiation survey reports conducted each month are provided to the local

TSA POC

100% of radiation survey reports required in a month are provided to the local TSA POC

Random Sampling of monthly requirement

CO may seek remedy in accordance with the FAR for missed performance

___________________________________________________________________________1-10

APPENDIX B: QUALITY ASSURANCE MONITORING REPORT FORM

QUALITY ASSURANCE MONITORING REPORT

LEVEL OF INSPECTION DATE COMPLETED

PERFORMANCE REQUIREMENT CONTRACT SECTION

PERFORMANCE STANDARD/ACCEPTABLE QUALITY LIMIT (AQL)

SURVEILLANCE METHOD Explain when evaluation is to occur and what is satisfactory or unsatisfactory.

NOTES

REQUIREMENT COMPLIANCE WITH PERFORMANCE

STANDARD/AQL

OTHER COMMENTS

Overall Rating Of Inspection

RATINGS OF INSPECTION : Exceptional (E), Satisfactory (S), Unsatisfactory (U)

Inspector Comments:

Contractor Representative Signature: Date: ____________________

QAE Signature: _______________________________________________ Date: ________

___________________________________________________________________________1-11

APPENDIX C: QUALITY ASSURANCE MONITORING LOG FORM

NOTE: FILL IN THE FIRST THREE COLUMS USING THE APPROPRIATE ENTRIES FROM THE PWS/CONTRACT

QUALITY ASSURANCE MONITORING ACTIVITIES

CONTRACT NUMBER EVALUATION PERIOD

CONTRACT/

PWS

SECTION

PERFORMANCE

REQUIREMENT

PERFORMANCE

STANDARD

ACCEPTABLE

QUALITY LIMIT

METHOD OF

SURVEILLANCE

COMPLIANCE

WITH

PERFORMANCE

STANDARD/

AQL

OTHER

COMMENTS

MONITORING

PERFORMED BY

& DATES

___________________________________________________________________________1-12

APPENDIX D: CONTRACT DISCREPANCY REPORT

CONTRACT DISCREPANCY REPORT

REPORT NUMBER

TO: (Service Provider and Manager Name)

FROM: (Name of COR)

DATES

PREPARED

ORAL NOTIFICATION RETURNED BY CONTRACTOR ACTION COMPLETE

DISCREPANCY OR PROBLEM (Describe in Detail. Include PWS references. Attach Continuation Sheet if necessary.)

SIGNATURE OF COR

TO: (Name of COR)

FROM: (Service Provider)

CONTRACTOR RESPONSE AS TO CAUSE, EFFECT, CORRECTIVE ACTION AND ACTIONS TO PREVENT RECURRENCE.

(Attach Continuation Sheet if necessary. Cite applicable CONTRACTOR QC program procedures or new QC procedures.)

SIGNATURE OF CONTRACTOR REPRESENTIVE

DATE

TSA EVALUATION (Acceptance, partial acceptance, or rejection. Attach Coordination Sheet if necessary.)

TSA ACTIONS (Cure notice, show cause, other.)

CLOSE OUT

NAME AND TITLE SIGNATURE DATE

CONTRACTOR

REP NOTIFIED

QAE

COR

___________________________________________________________________________1-13

APPENDIX E: ACRONYMS

AQL Acceptable Quality Limit CDR Contract Discrepancy Report CLS Contractor Logistics Support CO Contracting Officer COR Contracting Officer’s Representative FAR Federal Acquisition Regulation ILS Integrated Logistics Support OSC Office of Security Capabilities, TSA PM Preventative Maintenance PRS Performance Requirements Summary PWS Performance Work Statement QA Quality Assurance QASP Quality Assurance Surveillance Plan QC Quality Control QSP Quality System Plan TSRC TSA Service Response Center TSA Transportation Security Administration

3.4 Remedial Actions

4.2 Levels of Inspection