Attachment_4_QASP.doc

N00421-12-R-0013

N00421-12-R-0013

Quality Assurance Surveillance Plan (QASP)

For NAWCAD Naval Test Wing Atlantic (NTWL)

Squadrons VX-20, VX-23, HX-21, and Unites States Naval Test Pilot School (USNTPS), NAWCWD Naval Test Wing Pacific Test Squadrons VX-30 and VX-31 AIRCREW, ENGINEERING and OPERATIONS

1. Purpose

This Quality Assurance Surveillance Plan (QASP) is a Government developed and applied document used to make sure that systematic quality assurance methods are used in the administration of the performance-based contract. The intent is to ensure that the contractor performs in accordance with the performance metrics, the Government receives the quality of services required under the contract and that the actual performance results are documented in accordance with this plan.

This QASP describes how the Government will observe, evaluate and document the contractor’s performance in meeting the critical performance standards identified in the contract and task orders. Specifically, the QASP will identify the Government resources responsible for monitoring performance and surveillance methods that will be utilized in determining the quality level of services and products delivered under the contract. The intent is to establish and properly execute a QASP that will assist the Government in obtaining services from the contractor.

This contract requirement is for aircrew, engineering, and operations for Naval Test Wing Atlantic, VX-20, VX-23, HX-21, and the US Navy Test Pilot School and NTWP VX-30 and VX-31. The contract will be for one base-year and four (4) one-year options. A properly executed QASP will assist the Government in achieving the objectives of this procurement.

2.

Authority

Authority for issuance of this QASP is provided under Contract Section E – Inspection and Acceptance, which provides for inspections and acceptance of the services and documentation called for in PBSOW and this document, to be executed by the Contracting Officer or a duly authorized representative.

3.

Scope To fully understand the roles and responsibilities of the parties, it is important to first define the distinction in terminology between Quality Control Plan and the Quality Assurance Surveillance Plan. The contractor, and not the Government, is responsible for management and quality control actions necessary to meet the quality standards set forth by the contract.

The QASP on the other hand, is put in place to provide Government surveillance oversight of the contractor’s efforts to assure that they are timely, effective and are delivering the results specified in the contract.

4.

Government Resources The following definitions for Government resources are applicable to this plan:

Contracting Officer - A person duly appointed with the authority to enter into (Procuring Contracting Officer) (PCO) or administer (Administrative Contracting Officer) (ACO) contracts and make related determination and findings on behalf of the Government. The PCO for this contract is AIR-2.5.1.7. The ACO will be designated in the resulting contract. Contracting Officers are designated via a written warrant, which sets forth limitations of authority.

Contracting Officers Representative (COR) - An individual appointed in writing by the PCO to act as their authorized representative to assist in administering the contract. The COR will be appointed in the resulting contract. The limitations of authority are contained in a written letter of appointment.

5. Responsibilities

The following Government resources will have responsibility for the implementation of this QASP:

Contracting Officer – The Contracting Officer will monitor performance of all necessary actions for effective contracting to ensure compliance with the terms of the contract and safeguards the interests of the United States in the contractual relationship. It is the Contracting Officer that assures the contractor receives impartial, fair and equitable treatment under the contract. The Contracting Officer is ultimately responsible for the final determination of the adequacy of the contractor’s performance

Contracting Officer’s Representative – The COR is responsible for technical administration of the contract and assures proper Government surveillance of the contractor’s performance. In addition to the continuous monitoring of performance, the COR will also submit an annual assessment in the Navy’s Contractor Performance Assessment Reporting System (CPARS). For CPARS, the COR assesses general quality of the product or service, schedule, cost control, business relations, management, small business compliance, and other important areas. The COR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf. Any changes that the contractor deems may affect contract price, terms, or conditions shall be referred to the Contracting Officer for action.

6. Methods of QA Surveillance

a. Contractor Performance Assessment Report System (CPARS) – The market place for information technology, engineering, and management support services is very competitive. As such, the successful offeror has a vested interest in the Government generated CPARS rating under this contract. Additionally, an evaluation using the CPARS format will be performed at the midpoint of each period of performance. This evaluation will be one determinant in exercising an option. The Government assessment for this procurement will address the quality of product or service, schedule, cost control, business relations, management, and other important areas. As this information may affect future source selections throughout DoD, and the continuation of the contract, the annual Government assessment will be used appropriately as an additional performance oversight and communication tool with the QASP.

b. QASP - This QASP defines the surveillance process, responsibilities and acceptable quality standards that will apply to performance based contractual efforts and establishes the basic scoring and reporting methodology for surveillance documentation. The below listed methods of surveillance will be used by the COR in the technical administration of this QASP. In addition to the below instructions, the form to be used for documentation of quality assurance surveillance is the Surveillance Activity Checklist.

7.

Surveillance Surveillance shall be performed in accordance with the performance standards and incentives set forth herein. Enclosures (1), (2), (3) and (4) set forth the performance standards, definitions for evaluation areas and ratings, incentives, and activity checklist/surveillance methods for the contractor and COR. The ratings (exceptional, very good, satisfactory, marginal, or unacceptable) will be substantiated with a data analysis score of the contractor’s performance compared to the threshold and objective for each squadron as described in Enclosure (2).

8. Documentation

In addition to providing quarterly reports to the Contracting Officer, the COR will maintain a complete Quality Assurance file. The file will contain copies of all reports, evaluations, recommendations, and any actions related to the Government’s performance of the quality assurance function, including the originals of the Quality Assurance Checklists. All such records will be maintained for the life of the contract. The COR shall forward these records to the Contracting Officer at termination or completion of the contract.

9. Revisions to the QASP

Revisions are the joint responsibility of the COR and PCO. The COR and PCO, will review the QASP annually and will provide proposed changes to the Multi-Functional Team (MFT) for review if necessary. This document may be changed at any time upon review and approval by the MFT and supported by the PBSOW.

10. Enclosures

Enclosure (1) – PERFORMANCE STANDARDS

Enclosure (2) – DEFINITIONS FOR EVALUATION AREAS/RATINGS Enclosure (3) – INCENTIVES Enclosure (4) – ACTIVITY CHECKLIST

ENCLOSURE 1: PERFORMANCE-BASED QUALITY STANDARDS

Paragraph (2)

Conduct Category A and B, and when determined by the Commanding Officer, Category C and D flying and weapon system operation, flight test engineering, navigation, flight technical support, flight and ground tests, administrative flights, training flights, functional check flights, mission systems, performance, flying qualities, ordnance, HERO, EMC, EMI, and EMP , and project specialist support in fixed, rotary wing, unmanned and lighter-than-air air vehicles systems types, models and series provided by the government.

- 95% availability (# events completed/ missed)

- Documented attendance (Y/N)

- Government: Observation and assessment of report content for accuracy, relevance.

- Contractor monthly report: met commitment, description of content from event See enclosure (3)

- Monthly crew approval request (DCMA Form 644 or GFR approved equivalent)

- No consequential errors.

- Government observation

- Product inspection See enclosure (3)

The contractor shall operate the assigned aircraft, associated systems and instrumentation using proper and thorough preflight, in-flight, and post-flight procedures in compliance with the Naval Aviation Training and Operations Procedures Standardization (NATOPS) program or other flight operations procedures manuals provided by other DoD or commercial systems authorities for vehicles and systems under evaluation.

OPNAV 3710.7 series, NFM, NVD instruction, EMC/SOFT instruction

- All contractor furnished aircrew pass and exceed examination score criteria (SCORE)

- Zero mishaps with individual contractor furnished crew member as predominant causal contributor (Y/N) No lapse in currency, qualification caused by contractor

- Annual flight and instrument evaluation

- Government observation and report of real time flight performance

- Flight incident investigation Monthly crew approval request – GFR reporting records

Government flight operations logs and records See enclosure (3)

- Assignment for deliverables are documented in writing and attributed to the contractor. Drafts are ready for final approval.

- 100% of tasks completed (Y/N)

- Tasks not completed are documented and attributed to action nominated for subsequent correction by contractor (Y/N)

- Document drafts are usable close enough to final that no delay for re-write occurs.

- Government observation.

- Documented completion of written assignments presented by contractor.

- Inspection and approval by approving authority.

- Contractors monthly report (CDRL) See enclosure (3)

The contractor shall support test data collection compilation efforts by ensuring that data items collected are retrieved, annotated and delivered to test data analysis facilities. The contractor shall draft inputs for test progress reports per Government Furnished Format IAW CDRL A003.

- 100% of tasks completed (Y/N)

- Tasks not completed are documented and attributed to action nominated for subsequent correction by contractor (Y/N)

- Document drafts are usable close enough to final that no delay for re-write occurs.

- Government observation.

- Documented completion of written assignments presented by contractor.

- Inspection and approval by approving authority.

- Contractors monthly report (CDRL) See enclosure (3)

- All data assigned to contractor will be accounted for and evaluated.

- Deficiency reports (DR) will be accurate, complete, and accounted for in applicable deficiency tracking systems.

- Credibility (Y/N)

- 100% of data assigned addressed.

- 100% of DRs written will be logged for accounting.

-Initial DRs will be 90% complete for quick look analysis (data analysis can be immature for quick look)

- 100% of incomplete events documented with reason by contractor and reported at debrief

- Test event will be checked against data assignment by test director or designee (inspection)

- Post flight, test cards will be inspected by test director to insure quick look DRs are documented (inspection)

- DR tracking system check.

- DRs will be sampled for initial look.

See enclosure (3)

- All events scheduled for contractor involvement are attended

- All events are documented

- 90% of crew members scheduled make event.

- 100% of events are completed using back-up crew member

- Reports will be scored for credibility and accuracy.

- Inspection

- Observation

- Monthly report documenting content and attendance for assigned events

See enclosure (3)

- Reports are on time

- Compliant with specified format

- All elements are traceable to an active program. (Y/N)

- Reports submitted prior to 15th day (# days in advance)

- No errors in format or data requirements (Y/N

- Reports are on time

- Compliant with specified format

- All elements are traceable to an active program. (Y/N)

- Costs and hours reported are accurate to within whole dollars and 0.1 hours. (Y/N)

- Reports submitted prior to 15th day (# days in advance)

- No errors in format or data requirements (Y/N)

- Inspection

- Budget reviews by COR and BFMs See enclosure (3)

ENCLOSURE 2: DEFINITIONS FOR EVALUATION AREAS/RATINGS

2.1 This section describes the metrics used to measure contractor schedule and performance.

Quality of Product or Service - Assess the contractor’s effort to transform operational needs and requirements into an integrated solution. Areas of focus may include the planning and management of program task, the quality of support provided throughout all phases of contract execution, the integration of program management specialties, management of interfaces, and the management of a totally integrated effort of all program management concerns to meet cost performance, and schedule objectives. Assess how successfully the contractor meets program quality.

Schedule – Assess the contractor’s adherence to the required delivery schedule by assessing the contractor’s efforts during the assessment period that contribute to or effect the schedule variance. Also address significance of scheduled events (i.e., design reviews), discuss causes, and assess the effectiveness of contractor corrective actions.

Cost Control - Assess the Contractor’s effectiveness in forecasting, managing, and controlling contract cost. Is the contractor experiencing cost growth or under run? If so, discuss the causes and contractor-proposed solutions for the cost overruns. For contracts where tasks or contract sizing is based upon contractor provided person-hour estimates, the relationship of these estimates to ultimate cost should be assessed. In addition, the extent to which the contractor demonstrates a sense of cost responsibility, through the efficient use of resources in each work effort should be assessed.

Business Relations – Assess the timelines, completeness, and quality of a problem identification, corrective action plans, proposal submittals, the contractor’s reasonable and cooperative behavior, effective business relations, and customer satisfaction.

Management – Assess the contractor’s success with timely award and management of subcontracts, including whether the contractor met small/small disadvantages and women-owned business participation goals. Discuss the extent to which the contractor discharges its responsibility for integration and coordination of all activity needed to execute the contract; identifies and applies resources required to meet schedule requirements; assigns responsibility for tasks/actions required by contract; communicates appropriate information to affected program elements in a timely manner. Assess the contractor’s risk mitigation plans. If applicable, identify any other management areas that are unique to the contract.

Other areas – Assess additional evaluation areas unique to the contract or that cannot be captured elsewhere.

All efforts performed under the PBSOW and Government Furnished format reporting will be measured in accordance with the following evaluation rating.

Exceptional – Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the task and sub-task being assessed was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Very Good – Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the tasks and sub-task being assessed was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

Satisfactory – Performance meets contractual requirements. The contractual performance of the task and sub-task contain some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

Marginal – Performance does not meet contractual requirements. The contractual performance of the task and sub-task being assessed reflect a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory – Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractor’s performance of the task or sub-task contains a serious problem(s) for which the contractor’s actions appear or were ineffective.

ENCLOSUURE 3: INCENTIVES

The following incentives shall apply to performance under this contract.

Incentives

Base:

Not more than 2 of the measurement areas rated below “Satisfactory”. None of the measurement areas are rated below “Marginal”.

See below.

(-) Does not meet the acceptable performance definition as a condition for exercise of option 1.*

Option 1:

Not more than 2 of the measurement areas rated below “Satisfactory”. None of the measurement areas are rated below “Marginal”.

See below.

(-) Does not meet the acceptable performance definition as a condition for exercise of option 2.*

See below.

(-) Does not meet the acceptable performance definition as a condition for exercise of option 3.*

See below.

(-) Does not meet the acceptable performance definition as a condition for exercise of option 4.*

See below.

(-) Receive a negative final assessment (CPAR), appropriate for the quality of services provided.

The Government may not exercise an option unless all regulatory requirements are met and the contractor meets the acceptable performance definition.

Procedures

The Government COR or TPOC will perform inspections to ensure Contractor compliance with the appropriate paragraphs of the PBSOW, other Government requirements, and the Contractor’s own procedures, and will record the results of the inspections, noting the date and time. If inspection indicates a performance threshold is not met, the COR or TPOC will notify the Contractor and the contracting officer of the deficiency for correction. The Contractor shall be given a reasonable time after notification to correct the unacceptable performance. The length of time allowed to correct the problem will depend upon the requirement, complexity of the corrective action(s), and the deficiency. The Contractor will notify the COR or TPOC if the time allowed for correction is not adequate. The COR OR TPOC shall not consider the corrective action(s) complete until all deficiencies have been corrected and determined by the COR OR TPOC to be adequate (Government follow up). Contractor non-responsiveness, deficiencies not adequately addressed or corrected after notification of such, or estimated dates of completion that are not acceptable to the COR OR TPOC will be forwarded to the PCO for action.

Surveillance of Non-Service Summary Items

All tasks may be inspected on any appropriate basis, at a frequency determined primarily by the COR and/or TPOC, but also at the direction of the PCO.

Record Retention

The COR will maintain a file (a continuity book) which contains: documentation of results of all QA efforts including procedures review / verification, general area inspections, delegation letters, current contract/mods, QASP, training certificates, training documentation, CARs, CAR log, any correspondence with the Contractor, and any other beneficial communication for proper Contractor oversight. The continuity book will be kept for a minimum of 12 months after contract completion. Particular emphasis should be placed on deficiencies and corrective actions and procedural disapprovals. Surveillance records must be retained for a minimum of one year after completion of contract year and CPARS input.

Data Analysis

All inspection records, observations, monthly reports, CARs and other pertinent data will be reviewed monthly by the COR/TPOC to identify any trends in order to adjust surveillance activities. This does not preclude the necessity of adjusting surveillance due to specific incidents or deficiencies noted at any time. The review should include at a minimum a description of the data analyzed, a description of the conclusions/results, and actions planned/taken. These reviews will be documented and records maintained in the site continuity book.

Analysis of Results

At the end of the month, the COR will summarize all the information and compute the number of defects for each performance standard. The COR will determine if the Contractor’s performance for the specific standard is satisfactory or unsatisfactory based on an assessment of criticality of the defects. The contractor’s performance will be documented on the COR Monthly Report.

If one or more standard is unsatisfactory, further evaluation is required to determine if the Contractor's overall performance is unsatisfactory. The COR will assess the results of each performance standard, taking into consideration the importance and criticality of each in relation to the total service being evaluated. The COR will document the overall evaluation for the service on the Evaluation Summary.

Suggested Action

The Contractor should be notified immediately as to whether work performance has been satisfactory or not. If the Contractor's performance is unsatisfactory, a Corrective Action Request (CAR) should be issued and other actions, such as increasing the level of surveillance, may be considered.

Corrective Action Requests (CARs)

CARs may be issued verbally or as written findings. Verbal CARs are only applicable to minor deficiencies that can be corrected on the spot. Written CARs will be issued for repeat verbal CARs, when major deficiencies are detected, when minor deficiencies are noted that may affect system performance or the Contractor is clearly not performing to the contract requirements. Safety items and concerns automatically require the issuance of a written CAR, no questions asked.

The type and extent of corrective action to be taken depends on the nonconformance in terms of severity, frequency, and previous corrective actions. It is essential that the COR/TPOC pursue enforcement of all contractual requirements.

When contractual non-compliances are detected, corrective action must be initiated by the COR/TPOC utilizing QASP, Corrective Action Request (CAR) within 1 working day of discovery. A verbal notification of intent to submit a CAR must be made to NTWL GFR within 12 hours of discovery. Non-compliance issues should be resolved at the lowest possible level, but if the contractor is non-responsive, the COR/TPOC will escalate the CAR to a higher level of authority. The COR shall maintain a record of all CARs initiated (verbal and written, manual or electronic), action taken, and follow-up to verify implementation and effectiveness of the contractor actions. A Corrective Action Log shall be established for maintaining and tracking all Corrective Action Reports (CARs), issued to the contractor; All CARs shall be recorded and tracked by a control number.

A control numbering system shall be used, with the first twelve digits being the prime contract number, the next four digits will be the calendar year (CY) , next the activity/command and the last three digits will identify the corrective action request (i.e., XXXXXX-XX-TPS-001);

N00421-XX-C-XXXX - Contract Number TBA

CY - Calendar Year (i.e., 2011, 2012, 2013, etc.)

XXX – Command issuing the CAR

001 – First CAR for the Activity/Command for the fiscal calendar year

Note: Each activity shall use the following codes:

Air Test and Evaluation Squadron Twenty – VX-20

Air Test and Evaluation Squadron Twenty-One – HX-21

Air Test and Evaluation Squadron Twenty-Three – VX-23

Naval Test Pilot School – TPS

The COR shall document in the corrective action log the control number, the specific deficiency, contractor personnel with whom the deficiency was discussed, the date the deficiency was resolved and the follow-up actions taken by the COR to verify the contractor has ensured effectiveness of corrective actions taken.

The COR is required to maintain overall visibility of all Level II CARs and above. The originator of a CAR shall follow up to verify the implementation and effectiveness of the contractor’s actions and close out the CAR when appropriate. If the contractor is not responsive to lower-level CARs, the activity TPOC will notify the COR for escalation of the matter.

Levels of CARs

The four (4) CAR levels are as follows:

Level I

Level I CAR is when a contractual noncompliance requires no special management attention to correct. A Level I CAR may be directed to working level personnel. Level I CARs may be verbal or written (manual or electronic). These should not be routed to the Wing Staff, but must be logged in the CAR Log.

Level II

Level II CARs are written (electronic) requests for corrective action for contractual noncompliance that are systemic in nature, and could adversely affect cost, schedule, or performance if not corrected in a timely manner. Level II requests are directed to the contractor management level responsible for the process (group supervisor). Level II and above CARs must be routed to the Wing Staff.

Level III

Level III CARs involve escalation of serious contractual non-compliances to top contractor management, and may incorporate contractual remedies such as reductions of progress payments, cost disallowances, cure notices, show cause letters (manual or electronic), or business management systems disapprovals. Prior to issuance, a Level III CAR must be coordinated with the PCO. Level III CARs must be directed to contractor top management. Contractor top management may be notified by the PCO that continued inaction may lead to more severe measures.

A copy of Level III CARs and all subsequent correspondence must be provided to the buying activity. All affected contract administration team members (including DCAA and buying activities as appropriate) should be kept informed of the status and disposition of Level III requests.

Level IV

Level IV CARs involve contractual remedies such as suspension of progress payments or product acceptance activities, termination for default, and suspension or debarment, in accordance with FAR/DFARS policies and procedures. A Level IV CAR must be issued by the cognizant PCO and must be directed to contractor top management. A copy of the Level IV CAR and all subsequent correspondence must be provided to customer buying activities. Proper coordination and/or notification among Government activities of Level IV CARs are required.

When written CARs are issued the document must be specific in identifying the exact deficiency and the Contractor personnel with whom the deficiency was discussed. All written CARs will require the Contractor to identify the root cause of the deficiency, action taken to correct the specific deficiency, action taken to prevent recurrence, and the date when the corrective action will be implemented. The COR/TPOC will perform a follow-up audit to assure that the Corrective Action has effectively resolved the deficiency.

NAVAL TEST WING ATLANTIC (NAVTESTWINGLANT)

NAVAL TEST WING ATLANTIC (NAVTESTWINGLANT) has responsibility for the Contracting Officer (COR) functions, Government Quality Assurance functions, Government Flight Representative (GFR) functions. Points of Contact (POC) COR – TBD at contract award GFR – TBD at contract award Correspondence should be addressed to:

NAVAL TEST WING ATLANTIC

ATTN: TBD at contract award

22541 Millstone Road Bldg 304 R207

Patuxent River, MD 20670-1606 FAX Numbers: DSN - 342-7684, Commercial – 301-342-7684

Email and Phone contacts are: TBD at contract award

Reporting Requirements

The COR will provide copies of the Monthly COR Report to the Contractor. Contractors will have 10 calendar days to protest COR Status Report with the final administrative decision to be made by the PCO.

ENCLOSURE 4: SURVEILLANCE ACTIVITY CHECKLIST

Conduct Category A and B, and when determined by the Commanding Officer, Category C and D flying and weapon system operation, flight test engineering, navigation, flight technical support, flight and ground tests, administrative flights, training flights, functional check flights, mission systems, performance, flying qualities, ordnance, HERO, EMC, EMI, and EMP , and project specialist support in fixed, rotary wing, unmanned and lighter-than-air air vehicles systems types, models and series provided by the government.

Government: Observation and assessment of report content for accuracy, relevance.

- Contractor monthly report: met commitment, description of content from event

- Resume

- Monthly crew approval request (DCMA Form 644 or GFR approved equivalent)

TBA

- Product inspection

TBA

The contractor shall operate the assigned aircraft, associated systems and instrumentation using proper and thorough preflight, in-flight, and post-flight procedures in compliance with the Naval Aviation Training and Operations Procedures Standardization (NATOPS) program or other flight operations procedures manuals provided by other DoD or commercial systems authorities for vehicles and systems under evaluation.

- Annual flight and instrument evaluation

- Government observation and report of real time flight performance

- Flight incident investigation Monthly crew approval request – GFR reporting records

Government flight operations logs and records

TBA

- Documented completion of written assignments presented by contractor.

- Inspection and approval by approving authority.

- Contractors monthly report (CDRL)

The contractor shall support test data collection compilation efforts by ensuring that data items collected are retrieved, annotated and delivered to test data analysis facilities. The contractor shall draft inputs for test progress reports per Government Furnished Format IAW CDRL A003.

- Government observation.

- Documented completion of written assignments presented by contractor.

- Inspection and approval by approving authority.

- Contractors monthly report (CDRL)

TBA

- Post flight, test cards will be inspected by test director to insure quick look DRs are documented (inspection)

- DR tracking system check.

- DRs will be sampled for initial look.

TBA

- Observation

- Monthly report documenting content and attendance for assigned events

TBA

FOR OFFICIAL USE ONLY