NOVASTAR QASP.pdf

Quality Assurance Surveillance Plan (QASP)

NOVASTAR

20 January 2021

Version 1.0

1. Introduction

This Quality Assurance Surveillance Plan (QASP) has been developed to provide an effective and systematic method for monitoring, evaluating, and documenting contractor performance of the requirements in the Performance Work Statement of Work (PWS) at the Task Order Level. The Government will monitor contractor performance to assess the acceptability of the services provided and not the details of how the contractor accomplishes the work. The Government will use the QASP to determine if the contractor's performance meets the performance standards contained in the contract.

The QASP establishes procedures on how this assessment/inspection process will be conducted.

The Government retains the right to inspect all services and supplies furnished under the Indefinite Delivery/Indefinite Quantity (IDIQ) Task Orders in accordance with the provisions of the contract.

2. Authority

This QASP has been developed in accordance with FAR 46.4 and requiring agency instructions and regulations. Authority for issuance of this QASP is provided under Part 46 of the Federal Acquisition Regulation, Inspection of Services clauses, which provides for inspection, acceptance and documentation of the service called for in the contract or order. This acceptance is to be executed by the Contracting Officer (CO) or a duly authorized representative.

3. Contractor Responsibility

The contractor, and not the Government, shall be responsible for managing work, ensuring that performance is satisfactory and compliant with contract provisions and performing quality control and quality assurance functions to ensure that products and/or services meet task order requirements.

Additionally, the contractor shall be responsible for taking all actions necessary to correct unsatisfactory, deficient, or non‐compliant work. The QASP is not part of the contract nor is it intended to duplicate the contractor's quality control plan. This QASP is a living document. Flexibility in the QASP is required to allow for an increase or decrease in the level of surveillance necessary based on contractor performance.

4. Surveillance Monitoring

Contractor performance on task orders issued against the IDIQ will be monitored by the Chief Contracting Officer’s Representatives (CCORs), Contracting Officer’s Representatives (CORs) and Technical Subject Matter Experts (T‐SMEs)/Government Technical Monitors (GTM).

Surveillance methods will include review and analysis of end items, monitoring timeliness of deliveries, and observance of management practices and professionalism exhibited by contractor employees.

5. Roles and Responsibilities

NASICs CCORs, CORs, and T‐SMEs/GTM are responsible for monitoring, evaluating, and documenting contractor performance. They will work jointly to perform the following functions.

a. Discuss performance requirements with the contractor during the pre‐performance conference.

b. Establish a schedule for routine monitoring events, if applicable.

c. Review, monitor, and evaluate all items (services performed, work products, and/or delivered items) as listed on the Performance Assessment Report (PAR) in accordance with the stated performance measures. Verification of the level of performance compliance shall be performed by analysis, demonstration, inspection, or test. The evaluator shall determine the degree to which contractor performance meets the established performance standards.

d. Verify timeliness of deliveries.

e. Hold performance assessment meetings with the requiring agency’s technical representative.

f. Complete and/or compile performance documents for each evaluation period at Task Order Level.

g. Ensure that the narrative of all observations is accurate and factual in every respect. The narrative shall be legible and provide the evaluator’s observations, evaluation, and conclusions in precise descriptive language. Generalities, personal opinions and vague or ambiguous statements are not acceptable.

h. Provide an updated copy of the COR Monthly Performance Evaluation Report to the Contracting Officer upon completion of the evaluation via the Procurement Integrated Enterprise Environment (PIEE) Surveillance and Performance Monitoring (SPM) module.

i. Meet with the Contracting Officer to—

i. Report the results of contractor surveillance.

ii. Report the acceptance of services.

iii. Determine the Government’s plan of action in the event of unacceptable, unsatisfactory, deficient, or non‐compliant contractor performance.

j. Recommend needed changes to the QASP to the Contracting Officer and IDIQ Program Manager.

6. Surveillance Personnel

The following personnel shall oversee and coordinate surveillance activities.

1. IDIQ Program Manager

The NOVASTAR Program Manager (PM) enables acquisition planning and provides strategic acquisition and strategy guidance for all task orders placed on IDIQ. The PM provides primary program oversight, nominates the CCORs/CORs, ensures CORs are trained before performing any COR duties, and supports the COR’s performance assessment activities. The PM is responsible for ensuring task order surveillance is conducted and reported in accordance with Air Force and DoD guidance. While the PM may serve as a direct conduit to provide Government guidance and feedback to the contractor on technical matters, they are not empowered to make any contractual commitments or any contract changes on the Government's behalf.

Assigned PM: NASIC/XOQ

Organization or Agency: National Air and Space Intelligence Center, Wright‐Patterson AFB, OH

Telephone: 937‐522‐6285

Email: NASIC.XOQ.ORGBOX@us.af.mil

2. Technical SME / Government Technical Monitor

The T‐SME/GTM is the task order requirement owner and provides primary task order oversight. The COR uses the QASP to conduct the oversight/surveillance process. The T‐SME is not empowered to make any contractual commitments or any contract changes on the Government's behalf.

Assigned T‐SME/GTM: Assigned by Task Order

Organization or Agency: National Air and Space Intelligence Center, Wright‐Patterson AFB, OH

Telephone: Assigned by Task Order

Email: Assigned by Task Order

3. Contracting Officer (KO)

The Contracting Officer shall ensure performance of all necessary actions for effective contracting, ensure compliance with the contract terms, and shall safeguard the interests of the United States in the contractual relationship. The Contracting Officer shall also ensure that the contractor receives impartial, fair, and equitable treatment under this contract. Determine and document the final assessment of the contractor's performance in the Government’s past performance tracking system.

Assigned Contracting Officer: Assigned by Task Order

Organization or Agency: AFMC/AFLCMC/PZI

Telephone: Assigned by Task Order

Email: Assigned by Task Order

4. Chief Contracting Officer's Representative (CCOR)/Contracting Officer’s Representative (COR)

The CCOR will coordinate and provide oversight for all COR responsibilities and administrative duties including uploading monthly performance evaluations into the PIEE SPM module. CORs are responsible for ensuring continuous technical oversight of the contractor's performance is conducted by the T‐SME.

The COR uses the QASP to conduct the oversight/surveillance process. The COR shall keep a Quality Assurance file that accurately documents the contractor's actual performance. The COR will leverage the T‐SME requirements owners for the execution of on‐site surveillance. The purpose is to ensure that the contractor meets the performance standards contained in the contract. The COR is responsible for reporting early identification of performance problems to the KO. The COR is required to provide an annual performance assessment to the KO which will be used in documenting past performance. The QASP is the primary tool for documenting contractor performance. The COR is not empowered to make any contractual commitments or to authorize any contractual change on the Government's behalf.

Assigned CCOR: NASIC/XOQ (individual assigned by task order)

Assigned COR: Assigned by Task Order

Organization or Agency: National Air and Space Intelligence Center, Wright‐Patterson AFB, OH

Telephone: Assigned by Task Order

Email: Assigned by Task Order

5. Other Key Government Personnel

Information will be entered for each Task Order.

This may include performance monitors, inspectors, technical experts, or others who provide information that helps the COR monitor contractor performance.

6. Contractor Representatives

The following employees of the contractor serve as the contractor's Program Manager and Task Manager for this contract. (Complete this section after the contract award)

Program Manager ‐ <upon award, enter name>

Telephone: <enter number>

Email: <enter address>

Other Key Contract Personnel ‐ <upon award, enter name or delete these lines if not applicable>

Title: <enter title>

Telephone: <enter number>

Email: <enter address>

7. Performance Requirements and Method of Surveillance

This section describes the special requirements for this effort. The following sub‐sections provide details of various considerations on this effort.

7.1 Contract Surveillance

The goal of the QASP is to ensure that contractor performance is effectively monitored and documented. The COR's contribution is their professional, non‐adversarial relationships with the Contracting Officer, PM and the contractor, which enables positive, open and timely communications.

The foundation of this relationship is built upon objective, fair, and consistent COR evaluations of contractor performance against contract requirements. The COR uses the methods contained in this QASP to ensure the contractor is in compliance with contract requirements. The COR function is responsible for a wide range of surveillance requirements that effectively measure and evaluate the contractor's performance. Additionally, this QASP is based on the premise that the contractor, not the Government, is responsible for management and Quality Control/Quality Assurance (QC/QA) actions to successfully meet the terms of the contract.

7.2 Service Delivery Summary

A Service Delivery Summary (SDS) will be included with every task order to identify mission essential deliverables, performance objectives and standards that must be performed by the contractor. The SDS will detail the method of surveillance the COR will use to validate and inspect these performance elements. Inspection of each element will be documented in the COR file. A template SDS, to be tailored for each task order, is included as an attachment to this QASP.

Performance objectives define the desired outcomes. Performance Standards define the level of service required under the contract to successfully meet the performance objective. The inspection methodology defines how, when, and what will be assessed in measuring performance. The Government performs surveillance, using this QASP, to determine the quality of the contractor's performance as it relates to the performance element standards. The Performance Requirements Summary(PRS) should be used to form the foundation of the COR's inspection checklist.

7.3 Contractor Performance Assessment Reporting System (CPARS)

The CPARS is a web‐enabled application that collects and manages a library of automated contractor report cards. The CPARS report assesses the contractor’s performance and provides a record, both positive and negative, on a given contract for a specific period of time. The COR Personnel is required to complete a narrative summary of contractor past performance. The data will be input annually into the CPARS reporting system. COR Personnel will forward their CPARS assessment electronically to the CO who will review and add comments. The CPARS allows contractors to electronically submit comments regarding the Government’s assessment and indicate concurrence or non‐concurrence with the overall evaluation. . The contractor has 30 days to concur, non‐concur and/or make comments.

In evaluating the quality of contractor’s performance, the following performance ratings may be used.

The COR Personnel will support each rating with a written narrative summary.

Performance Rating Criteria

Exceptional Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub‐ element being assessed was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Very Good Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub‐ element being assessed was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

Satisfactory Performance meets contractual requirements. The contractual performance of the element or sub‐element contains some minor problems for which corrective actions taken by the contractor were satisfactory.

Marginal Performance does not meet some contractual requirements. The contractual performance of the element or sub‐element being assessed reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub‐element contains serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

The CPAR assessment will address the following elements:

Quality of Service: Assess the contractor’s conformance to contract requirements.

Schedule: Assess the timeliness of the contractor against contract requirements, task orders, milestones, delivery schedule and administrative requirements (e.g., efforts that contribute to or effect the schedule variance.)

Cost Control: Assess the contractor’s ability to remain within cost, accurately report and submit invoices and vouchers according to the terms of the task order/contract . The assessment will include a discussion of the contractor’s effectiveness in forecasting, managing and controlling contract cost, as applicable.

Business Relations. Assess the integration and coordination of all activity needed to execute the contract, specifically the completeness and quality of problem identification, corrective action plans, the contractor’s history of reasonable and cooperative behavior, customer satisfaction and management of subcontracts including progress on small business subcontracting goals and participation plans.

Management of Personnel: Assess the contractor’s performance in selecting, retaining, supporting and replacing personnel when necessary.

Regulatory Compliance: Adherence to regulatory and security requirements.

Surveillance Procedures: The performance objective and performance standards will dictate the minimum surveillance requirements the Government must accomplish. The COR may inspect any aspect of contractor performance at any time during the contract, whether or not it is included in the SDS.

Methods for Surveillance:

Customer Complaint: A method where the customer discovers unacceptable performance or a defect in service, and prepares a customer complaint record.

Periodic Surveillance. The COR will periodically assess the contractor’s performance by either watching actual performance, physically checking an attribute of the completed tasks, or inspecting the task or its results to determine whether or not the performance meets the standard contained in the contract.

100 % Inspection. In this surveillance method, the COR will inspect the contractor’s performance each time the task is performed.

Random Sampling. Random sampling is the most appropriate method for frequently recurring tasks. A typical use of this surveillance method is lot sampling. Initially, the COR will evaluate randomly selected samples of the activity to determine the acceptability of the entire requirement. Random sampling is statistically based and assumes the Government receives acceptable performance by the contractor if a given percentage or number of schedule inspections are found to be acceptable.

Remedies for Unacceptable Performances:

In accordance with the contract Inspection of Services clause, if any of the services do not conform to contract requirements; the Government may require the contractor to perform the services again in conformity with contract requirements; at no increase in contract amount. When the defects in services cannot be corrected by re‐performance, the Government may:

‐Require the contractor to take necessary action to ensure that future performance conforms to contract requirements; and

‐Reduce the contract price to reflect the reduced value of the services performed.

By contract or otherwise, perform the services and charge the contractor any cost incurred by the Government that is directly related to the performance of such service; or

‐Decrease the invoice amount in relation to performance, document negative contractor

CPAR ratings and negative PAR findings, issue cure notice, show cause notice and /or terminate the contract.

‐The CO will determine the action to be taken against the contractor, such as reductions in fee/price in accordance with the Inspection of Services clause. The CO’s disposition of the deficiency will be documented and the contractor will be notified of the action.

‐If the contractor is performing well, the contractor will receive the appropriate performance ratings and this may increase the possibility of receiving additional task order awards. However, when performance on an awarded task order is unacceptable, the continued negative performance will be documented within the contractor’s CPAR report, which may negatively influence the overall CPAR ratings.

7.4 Taking Corrective Actions

The Government will promptly notify the contractor of any unsatisfactory, unacceptable, deficient, non‐ conforming, or non‐compliant performance. The contractor shall have the opportunity to review the Government’s determination and provide comments.

Any contractor performance problems that result from the failure of the Government to fulfill any of its obligations under the contract or order, upon which contractor performance is dependent, will not be assessed and documented as contractor deficiencies or non‐compliance, to the extent of Government liability.

The contractor shall be responsible for correcting all unsatisfactory, unacceptable, deficient, or non‐ compliant performance. For firm fixed‐price contracts, the cost of re‐performing unsatisfactory, unacceptable, deficient, or non‐compliant work will be the sole responsibility of the Contractor. For Labor‐Hour contracts, the cost of re‐performing unsatisfactory, unacceptable, deficient, or non‐ compliant work will be reimbursed at the approved labor hour rate less an amount for profit, as prescribed by paragraph (f) of the clause entitle Inspection— Time‐and‐Material and Labor‐Hour (May 2001)(FAR 52.246‐6), which is included in this contract.

Disputes between the Contractor and the Chief CORs and CORs regarding surveillance results should be referred to the Contracting Officer.

Failure of the contractor to take appropriate and timely corrective action will result in the Government’s issuance of cure or show‐cause notices or pursue other remedies set forth in the provisions of the contract or as provided by law.

When unsatisfactory work is not corrected or unacceptable work is not re‐performed to the Government’s satisfaction, in addition to other available remedies, the Government may reduce the contract price to reflect the reduced value received. Price reduction limits are set forth in the attached Performance Evaluation Report. Additionally, the Government reserves the right to include any incidents of unsatisfactory, unacceptable, deficient, or non‐compliant performance, especially if uncorrected, in the contractor’s past performance record.

Revisions to the QASP:

Revisions to this QASP are the joint responsibility of the Contracting Officer and NASIC IDIQ Program Manager.

Appendix 1 – Service Delivery Summary TEMPLATE (All sections to be tailored for each task order and included in the task order PWS)

Statements Standards/AQLs Inspections

4.1 Program Management

Deliverables:

A001 Presentation Material A002 Report, Record of Meeting/Minutes A003 Management Plan A004 Status Report A005 Certificate of Services Technical Assistance Activity Report A006 Contract Work Breakdown Structure A007 Data Accession List (DAL) A008 Integrated Program Management Report A009 Implementation Plan A011 Manhour Estimate, Technical Cost Proposal A012 Contractor's Configuration Management Plan A013 System Engineering Management Plan

Quality: All reports accepted on first submittal.

AQL: Quality: 98% reports accepted on first submittal. Percentage based on reports submitted within the applicable fiscal year.

100% Inspection

What: Each Planning Document

Completeness of document contents in accordance with the data item description

Accuracy of the document contents

How: Each document will be reviewed when received.

Who: Contracting Officer's Representative Project or Task Manager Subject Matter Expert

Incentive: The contractor shall revise the document; addressing issues and comments raised by the Government reviewer. The contractor shall not charge the Government for revisions.

3.2 Research

Deliverables:

A001 Presentation Material A010 Technical Report Study/Service

a) Submit 6 innovation concepts (white papers) per year

AQL: Submit 3 innovation concepts (white papers) per year

100% Inspection

What: Delivery and recording of each white paper submittal

How: Review and evaluation

Who: Subject Matter Expert

Incentive: Evaluation of the number and quantity of the white papers will be used by the Government to determine award of future NOVASTAR options.

2.3 Development

Deliverables:

A001 Presentation Material A009 Implementation Plan A010 Technical Report Study/Service A014 Interface Control Document A015 Computer Program End Item Documentation A016 Software Development Plan A017 Software Installation Plan A018 Software Transition Plan A019 System/Subsystem Specification A020 System/Subsystem Design Description A021 Software Requirements Specification A022 Interface Requirements Specification A023 Software Design Description A024 Interface Design Description A025 Database Design Description A026 Software Test Plan A027 Software Test Description A028 Software Test Report A029 Software Product Specification A030 Software Version Description A031 Software User's Manual A032 Software Center Operator Manual A033 Test Procedure

a) Submit Total Cost of Ownership analysis ‐ 100% of designs AQL: Submit Total Cost of Ownership analysis ‐ 95% of designs

b) Quality: All reports accepted on first submittal.

AQL: Quality: 98% reports accepted on first submittal. Percentage based on reports submitted within the applicable fiscal year.

What: Total Cost of Ownership analysis

Each System Engineering Document

How: Review of Total Cost of Ownership Analysis

Content and completeness of all design documents

Who: Contracting Officer's Representative

Project or Task Manager

Subject Matter Expert

Program Manager

Government Systems Engineering Team

Incentive: The contractor shall revise the document; addressing issues and comments raised by the Government

A034 Security Test Plan A035 Acceptance Test Plan A036 Training Program Development and Management Plan A037 Training Situation Document A038 Instruction Performance Requirements Document A039 Instructional Media Requirements Document A040 Instructional Media Design Package A041 Training Program Structure Document A042 Course Conduct Information Package A043 Training Conduct Support Document A044 Training Evaluation Document A045 Test Package A046 Instructional Media Package A047 Training System Support Document A048 Web‐based Course A049 Commercial Off‐The‐Shelf Manuals and Associated Supplemental Data reviewer. The contractor shall not charge the Government for revisions.

Standard(s): Inspection applies to all standards

3.4 Sustainment

Corrective Maintenance

Deliverables:

A001 Presentation Material A010 Technical Report Study/Service A015 Computer Program End Item Documentation

a) System is 98% available

Mean Time to Repair is less than 4 hours

AQL: System is 95% available

Mean Time to Repair is less than 6 hours

Random Sampling

What: Review system logs

How: Plot availability and mean time to repair data from system logs

Time frame for requirement will be determined by the individual system specification

Who: Contracting Officer's Representative

Project or Task Manager

Subject Matter Expert

Program Manager

Government Systems Engineering Team

Standard(s): Inspection applies to all standards

3.5 Production

Deliverables:

A001 Presentation Material A010 Technical Report Study/Service

a) Quality: Time Critical Reports ‐100% of Essential Elements of Information complete AQL: Quality: Time Critical Reports ‐ 85% of Essential Elements of Information complete

What: Submitted Intelligence Reports and Products

How: Product compared with current standard for the item.

Who: Contracting Officer's Representative

Government Intelligence Analyst trained and qualified to inspect the product and determine accuracy of the information.

Incentive: Contractor shall revise the product to meet the standard without additional cost to the Government

Standard(s): Inspection applies to all standards

Appendix 2 ‐ Corrective Action Report (CAR)

CORRECTIVE ACTION REPORT (CAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACTOR

2. CONTRACT NUMBER

3. TYPE OF SERVICES

4. FUNCTIONAL AREA

5. SUSPENSE DATE

6. CONTROL NUMBER

7. DEFICIENCY MAJOR MINOR

FINDING:

FINDING IMPACT:

Please respond with a written corrective action plan that details the corrective action of the cited deficiency, the cause of the deficiency, and actions taken to prevent recurrence by Suspense Date in Block 5. If date was not entered in Block 5, the contractor is not required to provide a response.

8. QUALITY ASSURANCE PERSONNEL (COR)

TYPED NAME AND GRADE

SIGNATURE AND DATE

9. ISSUING AUTHORITY

TYPED NAME AND GRADE

SIGNATURE AND DATE

10. COR RESPONSE TO CONTRACTOR CORRECTIVE ACTION AND ACTION TAKEN TO PREVENT RECURRENCE

11. COR DETERMINATION

ACCEPTED REJECTED

12. CLOSE DATE

Appendix 3 ‐ Customer Complaint Record

CUSTOMER COMPLAINT RECORD

DATE/TIME OF COMPLAINT

SOURCE OF COMPLAINT

ORGANIZATION BUILDING

NUMBER

INDIVIDUAL PHONE NUMBER

NATURE OF COMPLAINT

CONTRACT REFERENCE

VALIDATION

DATE/TIME CONTRACTOR INFORMED OF COMPLAINT

ACTION TAKEN BY CONTRACTOR

RECEIVED/VALIDATED BY

Appendix 4 ‐ Performance Assessment Report (PAR)

PERFORMANCE ASSESSMENT REPORT (PAR)

(If more space is needed, use reverse and identify by number)

1. CONTRACT/TASK ORDER

NUMBER

2. CONTRACTOR 3. TYPE OF SERVICES

4. QUALITY ASSURANCE PERSONNEL (COR) SIGNATURE AND DATE

5. COR PHONE 6. SUSPENSE DATE

I. PERFORMANCE

7. DEFICIENCY (CHECK ALL BOXES THAT

APPLY)

NEW

REPEAT

NO DEFICIENCY NOTED

8. SERVICES SUMMARY or PWS PARAGRAPH ITEM

REVIEWED

9. BRIEF DESCRIPTION OF DEFICIENCY (IF DEFICIENCY BOX

WAS CHECKED)

10. DETAILED PERFORMANCE ASSESSMENT

II. CONTRACTOR VALIDATION

11. CONTRACTOR REPRESENTATIVE CONCUR

NON‐CONCUR

12. CORRECTIVE ACTION ESTIMATED COMPLETION DATE

13. CONTRACTOR REPRESENTATIVE CORRECTIVE ACTION AND PREVENTION OF RECURRENCE OR REASON FOR NON‐

CONCURRENCE OF COR CITED DEFICIENCY

III. ACTION CORRECTED

14. CONCUR NON‐CONCUR COR SIGNATURE AND DATE

15. COR REMARKS (REQUIRED)

6. CONTRACTOR REPRESENTATIVE REMARKS