ATT_2_QASP.docx

N00421-17-R-0065

Quality Assurance Surveillance Plan (QASP) For NAWCAD Naval Test Wing Atlantic (NTWL) Unmanned Aircraft Systems Test Directorate (UASTD)

1. Purpose

This Quality Assurance Surveillance Plan (QASP) is a Government developed and applied document used to make sure that systematic quality assurance methods are used in the administration of the performance-based contract. The intent is to ensure that the Contractor performs in accordance with the performance metrics, the Government receives the quality of services required under the contract and that the actual performance results are documented in accordance with this plan.

This QASP describes how the Government will observe, evaluate and document the Contractor’s performance in meeting the critical performance standards identified in the contract. Specifically, the QASP will identify the Government resources responsible for monitoring performance and surveillance methods that will be utilized in determining the quality level of services and products delivered under the contract. The intent is to establish and properly execute a QASP that will assist the Government in obtaining services from the Contractor.

This contract requirement is for aircraft maintenance and support services for Naval Test Wing Atlantic, Unmanned Aircraft Systems Directorate (UASTD). The contract will be for one base-year and Four (4) one-year options. A properly executed QASP will assist the Government in achieving the objectives of this procurement.

2. Authority

Authority for issuance of this QASP is provided under Contract Section E – Inspection and Acceptance, which provides for inspections and acceptance of the services and documentation called for in the Performance Based Statement of Work (PBSOW) and this document, to be executed by the Contracting Officer or a duly authorized representative.

3. Scope

To fully understand the roles and responsibilities of the parties, it is important to first define the distinction in terminology between Quality Control Plan (QCP) and the QASP. The Contractor, and not the Government, is responsible for management and quality control actions (QCP) necessary to meet the quality standards set forth by the contract.

The QASP on the other hand, is put in place to provide Government surveillance oversight of the Contractor’s efforts to assure that they are timely, effective and are delivering the results specified in the contract.

4. Government Resources and Responsibilities

The following Government resources and responsibilities are applicable to this plan:

4.1 Contracting Officer (CO) - A person duly appointed with the authority to enter into (Procuring Contracting Officer) (PCO) or administer (Administrative Contracting Officer) (ACO) contracts and make related determination and findings on behalf of the Government. The PCO for this contract is AIR-2.5.1.12. The ACO will be designated in the resulting contract. Contracting Officers are designated via a written warrant, which sets forth limitations of authority.

4.1.1 The CO will monitor performance of all necessary actions for effective contracting to ensure compliance with the terms of the contract and safeguards the interests of the United States in the contractual relationship. It is the CO that assures the Contractor receives impartial, fair and equitable treatment under the contract. The CO is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

4.2 Contracting Officer’s Representative (COR) - An individual appointed in writing by the PCO to act as their authorized representative to assist in administering the contract. The COR will be appointed in the resulting contract. The limitations of authority are contained in a written letter of appointment.

4.2.1 The COR is responsible for technical administration of the contract and assures proper Government surveillance of the Contractor’s performance. In addition to the continuous monitoring of performance, the COR will also submit an annual assessment in the Navy’s Contractor Performance Assessment Reporting System (CPARS). For CPARS, the COR assesses general quality of the product or service, schedule, cost control, business relations, management, small business compliance, and other important areas. The COR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf. Any changes that the Contractor deems may affect contract price, terms, or conditions shall be referred to the CO for action.

4.3 Government Flight Representative (GFR): The GFR leads the APT. GFR areas of responsibility are outlined in NAVAIRINST 3710.1. The GFR works in cooperative agreement with maintenance leadership and the APT to develop and implement a ground operations maintenance surveillance plan to perform active day-to-day surveillance of Contractor ground operations.

4.4 Maintenance Officer (MO): The MO is the head of the maintenance department and develops, establishes, and implements maintenance and material management policies and procedures to support naval aircraft, airborne weapons, attendant systems, and related support equipment.

4.4.1 The MO assists the GFR in reviewing the GOPs, detail sufficient Quality Assurance Representatives (QAR)/APT manning for program surveillance and provide subject matter expertise for conducting program audits as required, and provides comments on Corrective Action Requests (CAR) and Corrective Action Plans (CAP) issued by the APT in response to Contractor deficiencies.

4.5 Government Ground Representative (GGR): The GGR ensures contractor compliance with NAVAIRINST 3710.1G, execution of the APT Surveillance Plan supporting this QASP, assists the GFR in reviewing GOPs and maintains liaison with contractor management.

4.6 Aviation Program Team (APT): The APT consists of both government and military personnel who provide oversight of Contractor ground maintenance operations through regularly scheduled and unannounced audits, monitors, inspections and annual surveys.

5. Surveillance

5.1 This QASP defines the surveillance process, responsibilities and acceptable quality standards that will apply to performance based contractual efforts and establish the basic scoring and reporting methodology for surveillance documentation. The COR, and the APT will perform Government inspections, audits, and monitors to ensure Contractor compliance with the appropriate paragraphs of the PBSOW, other Government requirements, and the Contractor’s GOPs, and will record the results. If inspection indicates a performance threshold is not met, the Government will notify the Contractor and the CO, as applicable, of the deficiency for correction.

5.1.1 The Contractor shall be given a reasonable time after notification to correct the unacceptable performance. The length of time allowed to correct the problem will depend upon the requirement, complexity of the corrective action(s), and the deficiency. The Contractor will notify the squadron’s MO or GFR and/or COR if the time allowed for correction is not adequate. Corrective action(s) shall not be considered complete until all deficiencies have been corrected and determined to be adequate by the Government through follow up. Contractor non-responsiveness, deficiencies not adequately addressed or corrected after notification of such, or estimated dates of completion that are not acceptable to the squadron’s MO and/or GFR will be forwarded to the PCO via the COR for action.

5.1.2 Ground Operations Surveillance Plan. The GFR/GGR developed surveillance plan of Contractor ground operations related to the contractual requirements of the Ground and Flight Risk Clause (June 2010) in the contract.

5.2 The APT is responsible for monitoring the compliance and effectiveness of the Contractor’s safety and quality control programs. Any/all required Contractor QA inspections may be witnessed by APT personnel. In-process inspections are specific Contractor QA functions required during the performance of Contractor maintenance requirements and actions where satisfactory task performance cannot be determined after the task has been completed. These inspections include, but are not limited to, witnessing of: application of torque, functional testing, adjusting, assembly, servicing, and installation. Certain maintenance tasks, if improperly performed, could cause catastrophic equipment failure jeopardizing the safety of personnel. The QAR shall randomly select Safe for Flight (SFF) maintenance actions they oversee (SFF maintenance actions may be directed by the squadron’s MO). Items the QAR is responsible, at minimum, to verify include: Contractor QA was present for the required maintenance action(s); the torque wrench/calibrated tools/gauges (if required) are/is within their calibration interval and the stickers were not torn/missing; the maintenance publication was present and utilized; a FOD search was conducted prior to a panel closure, and tools were inventoried and accounted for. The surveillance of SFF items will be used to evaluate the quality and the safety of the work being performed.

5.3 Surveillance Methods:

5.3.1 Audits: The purpose of an Audit is to evaluate the acceptability of a Contractor’s process to meet requirements and/or desired results and/or to show procedural or contractual non-compliance. Audit results can be used to provide insight to where specific monitors and/or inspections should be conducted to determine process compliance, without conducting a complete repeat of the audit process. Quality auditing is concerned with three general types of audits.

-System Audits: A quality system audit is a documented activity performed in accordance with written procedures or checklists to verify (by examination and evaluation of objective evidence) that applicable elements of the quality assurance program have been developed, documented, and effectively implemented in accordance with specified requirements.

-Process Audits: An audit verifying/evaluating a manufacturing or test operation against a documented set of instructions and/or standards. These instructions and/or standards shall be used to measure conformance to standards and the effectiveness of instructions. Such an audit is used to validate conformity of a process, personnel and equipment such as time, temperature, pressure, composition, amperage, component mixture, etc. Process Audits are used for adequacy and effectiveness of the process controls over the equipment and operators as established by procedures, work instructions, and process specifications.

-Product Audits: A product audit is the examination, inspection or test of a product that has been previously accepted for the characteristics being audited. Such an audit is a re-inspection and retest of the product that has already been accepted or a review of documented evidence of acceptance. It includes performing operational tests to the same requirements used by manufacturing. The audit will measure the level of product conformance to specified standards of workmanship, performance, and quality. It is an indicator of quality going to the customer, and should be conducted from the customer's point of view. This type of audit frequently includes an evaluation of packaging; examination for cosmetic flaws; check for proper documentation and accessories such as proper tags, stamps, shipment preparation, and protection. The audit may involve a check of accuracy of the tests, test equipment, and test procedure.

5.3.2 Inspections: This method of surveillance will be directed by the MO. Inspections (normally conducted by the Contractor) require the greatest exercise of personal judgment on the part of the inspector. This type of inspection is generally for the purpose of detecting cracks, seams, missing paint, color finish, missing parts, holes not drilled, etc. Inspection is the method used to verify the physical characteristics of the product (size, weight, appearance, adherence to specified standards and engineering practices, quality of design, and construction) by examining the equipment in comparison with associated documentation. Inspection determines conformance to requirements without the use of special test equipment or analysis techniques. Inspections normally fall into one of the following categories:

-Receiving Inspections apply to material, components, parts, equipment, logs and records and documents. The inspections are normally conducted by Contractor QA to determine the condition of material, proper identification, maintenance requirements and disposition, and accuracy of accompanying records and documents.

-In-process Inspections are specific Contractor QA functions required during the performance of maintenance requirements and actions where satisfactory task performance cannot be determined after the task has been completed. These inspections include, but are not limited to, witnessing application of torque, functional testing, adjusting, assembly, servicing, and installation.

-Final Inspections are specific Contractor QA functions performed following the completion of a task or series of tasks. Inspections may be performed during any assembly stage of the product. An example of verification by inspection is measuring the physical dimensions of a chassis. Inspections are different from analysis in that inspections can typically be performed by QA whereas analysis usually requires system engineering.

-Monitors are a method used to spot check program processes vice conducting a complete audit. Questions are selected from the audit checklist to evaluate a Contractor’s use of their program guidance/process. The results can be used to detect positive and negative trends and whether or not specific programs need more surveillance than the required timelines.

-Demonstrations are a method used to verify that an item conforms to the specified requirements by observing the operation and functional performance of the product, or part of the product, generally without the use of special instrumentation to record quantitative measurements. This method is generally used when a requirement does not contain a specific numerical parameter that may be measured. Since no quantitative values are specified for demonstrations, pass/fail criteria are “yes/no” indications of functional performance. An example of verification by demonstration is verifying that a component provides status data.

6.0 Corrective Actions:

6.1 This QASP is a coordinated, transparent document that has been vetted and agreed to by both the Government and the Contractor. If, during the execution of this QASP, any Contractual Non-compliance is noted, the Contractor will be notified as soon as possible.

6.2 The MO or COR are authorized to reject all Contractor work not accomplished in accordance with the contractual requirements. The type and extent of corrective action to be taken depends on the nonconformance in terms of severity, frequency, and previous corrective actions. It is essential that all Government oversight personnel pursue enforcement of all contractual requirements. APT members are required to ensure the COR and GFR are keep abreast of any Contractor non-compliance issues associated with Contractor Procedures, the contract SOW or safety concerns. Formal documentation of Contractor non-compliance issues associated with ground and flight operations shall be initiated by the Corrective Action Request (CAR) process as delineated in NAVAIRINST 3710.1G, and as further directed in NAVTESTWINGLANT 3710.2.

6.3 Annual Survey Inspections. Annual Surveys and/or inspections, whether conducted by the UASTD APT, GFR or other outside Agency (Aircraft Controlling Custodians (ACC), NAVAIR 5.0, NTWL or Aviation Maintenance Management Team (AMMT) etc.) will be conducted each year contract(s) is/are in effect and work is being performed at UASTD. Surveys and inspections will be evaluated using, but not to, the following ratings:

-Exceptional Performance (Benchmark): The contractor meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the task and sub-task being assessed was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

- Very Good - Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the task and sub-task being assessed was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

- Satisfactory Performance (On-Track): Contractor meets contractual requirements. The contractual performance of the task and sub-task contain some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

- Marginal Performance (Needs Attention): The contractor does not meet contractual requirements. The contractual performance of the task and sub-task being assessed reflect a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented. (Note: This level of assessment may be characterized as Needs Attention (YELLOW) if discrepancies are pre-dominantly minor in nature but numerous in numbers or Needs Attention (ORANGE) if the assessment is based on a single critical discrepancy regardless of any number of minor discrepancies.

- Unacceptable Performance (OFF-TRACK): The contractor does not meet contractual requirements and recovery is not likely in a timely manner. The contractual performance of the task or sub-task contains a serious problem for which the contractor’s actions appear or were ineffective.

7.0 Unmanned Aircraft Test Directorate (UASTD) Maintenance Performance Standards

7.1 UASTD performs developmental flight-testing of UAS Systems to include flight test and evaluation of the air vehicle and its associated ground control stations. To gauge the contractor’s performance at meeting the goals necessary for UASTD to safely and effectively execute its testing mission(s) the following performance metrics will be used by the COR and CO as the means for evaluation.

7.1.1 Air Vehicle Material Condition/Awaiting Maintenance (AWM/Up-Gripes).

The maximum allowable daily “up-gripe” discrepancies on RA air vehicles shall be 5. For the purpose of this performance measure the following discrepancies will be excluded from the Up Gripe Target (UGT) count: valid Awaiting Parts (AWP), flyable Planning and Estimator (P&E) discrepancies, discrepancies awaiting Government disposition, Project Work Registers, TDs, and corrosion discrepancies, High Time Components, Post Maintenance awaiting flight or turn discrepancies (FCF requirements, TQ checks, leak checks, etc.), awaiting AIMD maintenance, and authorized relief discrepancies as approved by the MO

Along with the discrepancy exceptions cited above, aircraft with more than one special inspection that are graded will only be graded for one discrepancy. For example, BUNO 072 has 7 day, 14 day and 28 day inspections due, and although there are three outstanding inspections, combined they will only count as one up-gripe Each RA air vehicle, as determined by the Aircraft Maintenance Supply Readiness Report (AMSRR), will receive a Daily UGT score. Scores will be determined and recorded per the following table:

UGT count will be determined by screening NALCOMIS Aircraft Equipment Workload Reports or ADBs each day.

Each UAS will receive a Monthly UGT Average.

Monthly UGT Average of all air vehicles will be calculated and reported monthly per the following table:

For the purposes of this metric, normal mathematical rounding rules apply.

***This Metric will be weighted as 50% of the Performance Metric on the contractor’s monthly performance measurement/report for UASTD***

7.1.2 Air Vehicle Material readiness Condition/Inspections.

The average number of uncorrected discrepancies for each aircraft completing a major (as determined per T/M/S by the UASTD MO) inspection shall not exceed 5, excluding valid Awaiting Parts (AWP), flyable Planning and Estimator (P&E) discrepancies, Project Work Registers, discrepancies awaiting Government disposition, and Depot Level TDs, Instrumentation Modifications. Corrosion discrepancies shall be corrected prior to aircraft being released from a major inspection.

UASTD APT personnel will review all aircraft completing major inspections for uncorrected discrepancies.

Contractor performance will be measured, graded, and reported monthly.

- To receive a grade of 1, the Contractor must achieve no more 2 uncorrected discrepancies for all aircraft completing a major inspection.

- To receive a grade of 0, the Contractor must achieve an average of no more than 5 uncorrected discrepancies for all aircraft completing a major inspection.

- To receive a grade of -1, the Contractor must achieve an average of more than 6 uncorrected discrepancies for all aircraft completing a major inspection.

***This Metric will be weighted as 50% of the Performance Metric on the contractor’s monthly performance measurement/report for UASTD***.

7.1.3 Daily Flight Schedule (DFS) Commitment:

The contractor shall provide a status of air vehicles and GCS stations to UASTD by 0730 of each day. The Government will provide the Contractor a draft daily flight and ground event schedule by 1200 hours each day for review and recommendations for next day’s events. The Contractor shall ensure all aircraft scheduling commitments made to the DFS are met. All aircraft maintenance and flight preparations (including ordnance evolutions and SE requirements) shall be complete, and the aircraft shall be in-place 1 hour prior to the scheduled event.

UASTD APT personnel will grade and record the Contractor’s performance on a daily basis per the following:

For the purposes of this schedule metric, changes made to the Daily Flight Schedule after the prior day’s flight schedule has been signed, whichever occurs first, will not be graded or recorded.

For the purposes of this schedule metric, changes made to the Daily Flight Schedule without the Contractor’s knowledge or recommendation will not be graded or recorded.

For the purposes of this schedule metric, downing discrepancies found on the first event of the day that are covered under the aircraft Daily/Turn-Around inspection requirements shall constitute a failure to meet the DFS Commitment Metric for that event. Downing discrepancies found on subsequent events that are covered under the applicable Turn-Around inspection requirements shall constitute a failure to meet the DFS Commitment Metric for that event.

For the purposes of this schedule metric, aircraft reported as meeting the DFS commitment that subsequently abort the Flight/Ground event due to a repeat discrepancy shall be corrected in reporting as Failed To Meet the DFS Commitment.

***This Metric will be weighted as 100% of the Schedule Metric on the contractor’s monthly performance measurement/report for UASTD. ***

SOW TASK DESCRIPTION

PERFORMANCE STANDARD AND ACCEPTABLE QUALITY LEVEL (AQL)

SURVEILLANCE METHOD

INCENTIVES

Maintain / correct aircraft material condition and conduct quality programs to ensure safe flyable aircraft

Maintain material condition of aircraft during major inspections by having no more than 5 uncorrected discrepancies at completion of inspection.

UASTD will review aircraft that have completed a major inspection and calculate and record outstanding discrepancies

See Enclosure (1)

Maintain / correct aircraft material condition and conduct quality programs to ensure safe flyable aircraft

Maintain material condition of aircraft by having less than 5 corrosion discrepancies at completion of Inspection.

UASTD will inspect/review and calculate and record 25% of all aircraft corrosion discrepancies.

Maintain / correct aircraft material condition and conduct quality programs to ensure safe flyable aircraft to meet Daily Flight Schedule Commitment (DFS)

Ensure aircraft on line and configured for flight/ground event 1 hour prior to scheduled event to meet DFS commitments by attaining daily scores of 1 at no less than 90% (based on DFS formula)

UASTD records daily and averages for the month

7.2 Performance Metrics

8.0 Reporting Requirements and Record Retention

8.1 The Monthly Status Report shall be submitted by email by the 10th day of the following month. The overall order schedule and performance scores will be a simple average of the schedule and performance scores, respectively, from the individual squadrons. Contractors will have 10 calendar days to protest COR Monthly Status Report scores with the final administrative decision to be made by the PCO.

9.0 Data Analysis

9.1 All inspection records, observations, monthly reports, CARs and other pertinent data will be reviewed monthly by the COR/MO to identify any trends in order to adjust surveillance activities. This does not preclude the necessity of adjusting surveillance due to specific incidents or deficiencies noted at any time. The review should include at a minimum a description of the data analyzed, a description of the conclusions/results, and actions planned/taken.

- Analysis of Results: at the end of the month, the MO will summarize all the information and compute the number of defects for each performance standard. The MO will determine if the Contractor’s performance for the specific standard is satisfactory or unsatisfactory based on an assessment of criticality of the defects. The contractor’s performance will be documented on the Monthly COR/MO Monthly Report.

- If one or more standard is unsatisfactory, further evaluation is required to determine if the Contractor's overall performance is unsatisfactory. The MO will assess the results of each performance standard, taking into consideration the importance and criticality of each in relation to the total service being evaluated. The MO will document the overall evaluation for the service on the Evaluation Summary. The Monthly Report will be reviewed and signed by the Squadron MO then forwarded to the NTWL COR.

10.0 Revisions to the QASP

Revisions are the joint responsibility of the COR, UASTD and PCO. The COR, UASTD and PCO, will review the QASP annually and will provide proposed changes to the contractor and other government entities for review if necessary. This document may be changed at any time upon review and approval by the COR, PCO and/or ACO and supported by the PBSOW.

Enclosure (1): Incentives The following incentives shall apply to performance under this contract.

Incentives

Base:

Not more than 2 of the measurement areas rated below “Satisfactory”. None of the measurement areas are rated below “Marginal”.

See below.

Midpoint of the assessment period (6 month) evaluation using the CPARS format covering the previous 6 months.

(+) Meet the acceptable performance definition as a condition for exercise of option 1.* (-) Does not meet the acceptable performance definition as a condition for exercise of option 1.*

(-) Receive a negative final assessment (CPAR), appropriate for the quality of services provided.

The Government may not exercise an option unless all regulatory requirements are met and the contractor meets the acceptable performance definition.

FOR OFFICIAL USE ONLY