Attachment_7-_QASP.docx

5.4.3 Engineering and Tech. Services QASP

N00421-15-R-0011 Quality Assurance Surveillance Plan (QASP) For the Simulation Division (5.4.3) Engineering Support Services Single Award Task Order Contract

1. Purpose This Quality Assurance Surveillance Plan (QASP) is a Government developed and applied document used to ensure systematic quality assurance methods are used in the administration of the Performance Work Statement (PWS) under the planned contract. The intent is to ensure that the Contractor performs in accordance with the performance metrics identified herein and the Government receives the quality of services called for in the order.

This requirement is for engineering and technical support services for the Simulation Division (NAWCAD 5.4.3) teams, infrastructure, and supported projects. These support services include project management, project support, hardware engineering, software development and engineering, and specialized technical support.

This performance-based Single Award Task Order Contract will have Cost Plus Fixed Fee (CPFF) labor CLINs and Cost CLINs for Other Direct Charges (ODCs) (Material and Travel). The contract will be for a three-year (36 month) ordering period. A properly executed QASP will assist the Government in achieving the objectives of this procurement.

2. Authority Authority for issuance of this QASP is provided under Federal Acquisition Regulation (FAR) Subpart 46.4 – Government Contract Quality Assurance, which requires inspections and acceptance of the services and documentation called for in service contracts and this contract, to be executed by the Contracting Officer or a duly authorized Government representative.

3. Scope To fully understand the roles and responsibilities of the parties, it is important to first define the distinction in terminology between Quality Control Plan and theQASP. Within a Quality Control Plan, the Contractor, and not the Government, is responsible for management and quality control actions necessary to meet the quality standards set forth by the contract..

The QASP is put in place to provide Government surveillance oversight of the Contractor’s efforts to assure that they are timely, effective and are delivering the results specified in the order.

4. Government Resources The following definitions for Government resources are applicable to this plan:

Contracting Officer - A person duly appointed with the authority to enter into contracts (Procuring Contracting Officer -PCO) or administer contracts (Administrative Contracting Officer - ACO) and make related determination and findings on behalf of the Government. The PCO for this contract is AIR 2.5.1.6. The ACO will be designated in the resulting contract. Contracting Officers are designated via a written warrant, which sets forth limitations of authority.

Contracting Officer’s Representative (COR) - An individual appointed in writing by the PCO to act as their authorized representative to assist in administering the contract. The COR will be appointed by written letter of appointment which will set forth limitations of authority.

Alternate Contracting Officer’s Representative (ACOR) - An individual appointed in writing by the PCO to act as their authorized representative, in the absence of the COR, to assist in administering the contract. The ACOR will be appointed in the resulting contract. The limitations of authority are contained in a written letter of appointment.

Technical Point of Contact– The COR designates an individual Government Project Lead to assist in administering a specific project delineated at the Task Order Level. .

5. Responsibilities

The following Government resources shall have responsibility for the implementation of this QASP

Contracting Officer – The Contracting Officer ensures performance of all necessary actions for effective contracting, ensures compliance with the terms of the contract, and safeguards the interests of the United States in the contractual relationship. It is the Contracting Officer that assures the Contractor receives impartial, fair and equitable treatment under the contract. The Contracting Officer is ultimately responsible for the final determination of the adequacy of the Contractor’s performance.

Contracting Officer’s Representative (COR) – The COR is responsible for technical administration of the contract and assures proper Government surveillance of the Contractor’s performance. The COR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf. Any changes that the Contractor deems may affect contract price, terms or conditions shall be referred to the Contracting Officer for action.

Alternate Contracting Officer’s Representative (ACOR) – The ACOR is responsible for technical administration of the contract and assures proper Government surveillance of the Contractor’s performance in the absence of the COR. The ACOR is not empowered to make any contractual commitments or to authorize any changes on the Government’s behalf. Any changes that the Contractor deems may affect contract price, terms or conditions shall be referred to the Contracting Officer for action.

Technical Point of Contact– Technical Point of Contact is responsible for assisting in administering a specific project under the contract. A Government Project Lead cannot, in any manner, alter the scope of the contract, make commitments or authorize any changes on the Government’s behalf .

6. Methods of QA Surveillance

a. a. Contractor Performance Assessment Report System (CPARS) – The Government will perform an evaluation that addresses quality, schedule, cost control, management, regulatory compliance, utilization of small business, and other important areas using the CPARS format annually covering the previous 12 months. The marketplace for integration, design and technical support services is very competitive; as such, the successful offeror has a vested interest in the Government generated CPARS under this contract. As this information may affect future source selections throughout DoD, and future orders, the annual Government assessment will be used appropriately as an additional performance oversight and communication tool.

b. QASP - The below listed methods of surveillance shall be used by the COR and Technical Point of Contact in the technical administration of this QASP. In addition to the below instructions, the form to be used for documentation of quality assurance surveillance is the Surveillance Activity Checklist provided as Enclosure (4) herein.

c. Observation, Feedback, and Review – Unscheduled, on-site observation of products and services provided by the Contractor, and recurring review of Contractor submitted reports and documentation identified in the CDRLs will be performed by the COR and ACOR. Feedback from various program customers will be collected and used by the COR to document surveillance of technical performance.

7. Surveillance

a. Monthly QA Surveillance Activities The COR will provide monthly quality assurance surveillance of the contract activities culminating with the submission of the Monthly COR Report and Surveillance Activity Checklist within 15 days of the reporting month. As this is a single award task order contract, monitoring of individual task orders will be conducted on an ongoing basis. However, formal surveillance documentation will be completed at the contract level, taking into consideration all efforts being performed on each task order in the respective evaluation period Monthly quality assurance surveillance will include the following activities:

1) Financial Review

a. Review of the Contractor’s Monthly Financial Summary (CDRL A001) and Project Financial Summaries (CDRL A002)

i. Compare funding listed with BFM funding report.

ii. Check for and investigate potential out-of-place or unexpected charges.

iii. Review the funding, expenditures, and burn-rate of each CLIN. Investigate unexpected burn rate values. Identify potential funding shortfalls.

iv. Review funding, expenditures and burn-rates for each project/activity supported by the contract. Communicate future funding requirements to project leaders.

v. Investigate any concerns identified by project Technical Points of Contract (TPOC) after they have reviewed their Project Financial Summaries for accuracy.

b. Investigate over-time charges. Ensure all over-time is allowable and reasonable (see contract clause 52.222-2).

c. Download Invoice documents and compare with Monthly Financial Report (CDRL A001). Note: contractor’s invoice period will not coincide exactly with monthly reporting periods, and sub-contractor charges are often delayed.

d. Record and plot historical monthly burn rates for all CLIN’s and for significant projects. Identify and rectify deviations with current spend plans.

2) Review Monthly Progress Report (CDRL A002)

a. Identify and investigate identified program and project concerns and risk areas.

b. Review for work that may be outside the scope of the contract PWS.

c. Review for unexpected and/or unauthorized work.

3) Performance Review

a. Discuss with Project Leaders the perceived quality, schedule and cost of services provided to their projects thru this contract.

b. Discuss with Simulation Division Management the perceived quality, schedule and cost of services provided to the organization thru this contract.

c. Review perceived overall performance with contractor. Discuss performance risks and potential paths for improving performance by the contractor and/or government.

4) Evaluation

a. Complete monthly Surveillance Activity Checklist (see Enclosure 4)

b. Complete and submit Monthly COR Report

b. Yearly Surveillance Activities Yearly Surveillance Activities will include completion of the Contractor Performance Assessment Reports System (CPARS).

Enclosures (1), (2), and (3) sets forth the performance standards, incentives, and surveillance methods for the contractor and COR while enclosure (4) provides the checklist and means for the COR to document on a monthly basis.

8. Documentation The COR shall submit the Surveillance Activity Checklist, Enclosure 4, with the Monthly COR report electronically by posting them to the CORT Tool within Wide Area Work Flow (WAWF) (https://wawf.eb.mil/), where they will be reviewed and approved by the Contracting Officer. Copies of the COR Monthly Reports, Surveillance Activity Checklists, and CPARS evaluations will be maintained by the COR in a Quality Assurance File. All such records will be maintained for the life of the Contract. The COR shall forward these records to the Contracting Officer at termination or completion of the Contract.

9. Enclosures Enclosure (1) –Performance Standards Enclosure (2) - Data Performance Standards Enclosure (3) - Incentives Enclosure (4) - Surveillance Activity Checklist

Enclosure 1

ENGINEERING AND TECHNICAL SUPPORT SERVICES PERFORMANCE STANDARDS

1) Compliance with all applicable Simulation Division (5.4.3) Quality Management System (QMS) standards and Procedures 100% of the time.

2) Compliance with all applicable security standards and procedures 100% of the time.

Program and Project Management Support PWS Section 4.2 Program management acceptable quality level is:

1) Program requirements met 95% of the time.

2) CDRL reports complete and on time in accordance with enclosure 2.

Project Support PWS Section 4.3 Project Support acceptable quality Level is:

1) Configuration management of data processed, tracked, and reported accurately 98% of the time.

2) Logistics documentation and reports accurate 95% of the time and complete 90% of the time upon initial complete submission.

3) Project requirements – as submitted by project team - complete, accurate and traceable 99% of the time.

4) Product test plans, executions and reports accurate and relevant to the minimization of product discrepancies 90% of the time.

Hardware Engineering Support PWS Section 4.5 Hardware Engineering acceptable quality level is:

1) Hardware and electrical engineering designs successfully meet established requirements without design revision 95% of the time.

2) 90% of design drawings are not revised due to design error at the completion of product fabrication.

3) 80% of design drawings are not revised due to drafting error or drafting oversight at the completion of product fabrication.

Software Development and Engineering Support PWS Section 4.6 Software Development and Engineering acceptable quality level is:

1) Software meets defined customer and derived requirements 97% of the time prior to formal product testing.

2) Software support documentation and notes are accurate and complete to a degree that an experienced engineer or programmer can understand the code’s purpose, structure, and implementation within a few days and be able to modify and work with the code.

3) Software is modular and reusable to the maximum extent possible with 100% accurate interface documentation at release.

Specialized Technical Support PWS Section 4.7 Specialized Technical Support acceptable quality level is:

1) Subject Matter Experts (SME), including scenario development experts, capable of providing the expertise required 95% of the time.

2) Plan, execute and document VV&A efforts accurately and on schedule 90% of the time.

3) Visual and sensor databases and products meet or exceed 97% of established customer requirements prior to formal product testing.

Enclosure 2

DATA PERFORMANCE STANDARDS

CDRL A001:

Monthly Financial Summary

1) Accurate and complete 98% of the time.

2) On time delivery 90% of the time.

3) Delivery is never more than 7 days late.

CDRL A002:

Monthly Project Financial Summary

4) Accurate and complete 98% of the time.

5) On time delivery 90% of the time.

6) Delivery is never more than 7 days late.

Project Financial Reports 100% Inspected by COR and Government TPOC, who will report any concerns to COR

CDRL A003:

Monthly Progress Report

1) Accurate and complete, providing an overview of the work accomplished during the reporting period.

2) Information is organized in a logical format and presents an appropriate level of detail to communicate Program status and accomplishments.

3) On time delivery 90% of the time.

4) Delivery is never more than 7 days late.

100% Inspection by COR

CDRL A004:

OPSEC Plan

CDRL A005:

Additional Reporting

Enclosure 3

INCENTIVES

All incentives are based on CPARS evaluations of the contracted effort. CPARS reports will be submitted for the entirety of the contract effort, and not individually for each Task Order. CPARS evaluations will be performed every 12 months thereafter, with a final CPARS report submitted at the end of the contract period of performance. CPARS evaluations will be based on the monthly and annual surveillance checklists (see Enclosure 4) and contract performance information collected by the COR.

CPARS evaluations asses the performance of a contract using six performance measurement areas:

Quality – Assess the achieved product performance relative to performance parameters required by the contract/order. Assess the contractor's effort to transform operational needs and requirements into an integrated system design solution. Areas of focus should be the planning and control of technical program tasks, integration, management of interfaces, interoperability, and the management of a totally integrated effort to meet cost, technical performance, and schedule objectives. The evaluation should cover these disciplines: systems architecture, design, manufacturing, integration and support, configuration control, documentation, test and evaluation. The evaluation for test and evaluation should consider success/problems/failure in developing test and evaluation objectives; planning (ground/air/sea) test, simulations and/or demonstrations; in accomplishing those objectives and on the timeliness of coordination and feedback of the test results (simulations/demonstrations) into the design and/or manufacturing process. Other activities include: producibility engineering, logistics support analysis, supportability considerations (maintenance personnel/skills availability or work-hour constraints, operating and cost constraints, allowable downtime, turn-around-time to service/maintain the system, standardization requirements), survivability, human factors, reliability, quality, maintainability, availability, inspectability, etc.

Assess the success of the contractor's performance in accomplishing logistics planning. For example, maintenance planning; manpower and personnel; supply support; support equipment; technical provisioning data; training and support; computer resources support; facilities; packaging, handling, storage and transportation; design interface; the contractor's performance of logistics support analysis activities and the contractor's ability to successfully support fielded equipment. When the contract/order requires technical and/or engineering data deliverables, the cognizant cataloging and/or standardization activity comments should be solicited.

Assess how successfully the contractor meets program quality objectives; e.g., producibility, reliability, maintainability, inspectability, testability, and system safety, and controls the overall process. The PM or contracting officer must be flexible in how contractor success is measured, e.g., data from design test/operational testing successes, field reliability and maintainability and failure reports, user comments and acceptance rates, and improved subcontractor and vendor quality.

Assess all the other technical activity critical to successful contract/order performance. Identify any additional evaluation aspects that are unique to the contract/order or that cannot be captured in another sub-element.

Schedule – Assess the timeliness of the contractor against the completion of the contract, task orders, milestones, delivery schedules, and administrative requirements (e.g., efforts that contribute to or affect the schedule variance).

This evaluation of the contractor’s adherence to the required delivery schedule should include the contractor’s efforts during the evaluation period that contribute to or affect the schedule variance. Also, address significance of scheduled events (e.g., design reviews), discuss causes, and assess the effectiveness of contractor corrective actions.

This element applies to contract/order closeout activities as well as contract/order performance. Instances of adverse actions such as the evaluation of liquidated damages or issuance of Cure Notices, Show Cause Notices, and Delinquency Notices is indicators of problems, which may have resulted in variance to the contract/order schedule and should be noted in the evaluation.

Questions to consider include the following:

Cost Control – Assess the contractor’s effectiveness in forecasting, managing, and controlling contract/order cost. If the contractor is experiencing cost growth or underrun, discuss the causes and contractor-proposed solutions for the cost overruns or underruns. For contracts/orders where task or contract sizing is based upon contractor-provided person hour estimates, the relationship of these estimates to ultimate task cost should be assessed. In addition, the extent to which the contractor demonstrates a sense of cost responsibility, through the efficient use of resources, in each work effort should be assessed. Include, as applicable, the following information:

Management – Assess the integration and coordination of all activity needed to execute the contract/order, specifically the timeliness, completeness and quality of problem identification, corrective action plans, proposal submittals, the contractor’s history of reasonable and cooperative behavior (to include timely identification of issues in controversy), customer satisfaction, timely award and management of subcontracts. Include, as applicable, information on the following:

Regulatory Compliance – Assess compliance with all terms and conditions in the contract/order related to applicable regulations and codes. Consider aspects of performance such as compliance with financial, environmental (example: Clean Air Act, Clean Water Act), safety, and labor regulations as well as any other reporting requirements in the contract. Consider questions such as the following:

Has the contractor complied with all contract clauses and reporting requirements (e.g., FAPIIS, FAR Subsection 9.104-6, FAR Clause 52.209-9)?

Other areas – Assess additional evaluation areas unique to the contract or that cannot be captured elsewhere.

Each CPARS performance measurement area is given one of five standard ratings:

Exceptional – Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the task and sub-task being assessed was accomplished with few minor problems for which corrective actions taken by the contractor were highly effective.

Very Good – Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the task and sub-task being assessed was accomplished with some minor problems for which corrective actions taken by the contractor were effective.

Satisfactory – Performance meets contractual requirements. The contractual performance of the task and sub-task contain some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

Marginal – Performance does not meet contractual requirements. The contractual performance of the task and sub-task being assessed reflect a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

Unsatisfactory – Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the task or sub-task contains a serious problem(s) for which the contractor’s actions appear or were ineffective.

Numeric values have been assigned to the five standard ratings to assist in calculating weighted average ratings of each of the six performance measurement areas based on ratings of several contract requirements.

The following incentives shall apply to performance under this contract:

Incentives

Ordering Period (36 months) :

(-) Does not meet the acceptable performance definition. CPARS documented accordingly.

Enclosure 4

SURVEILLANCE ACTIVITY CHECKLIST

Monthly Surveillance:

The COR will perform a monthly assessment of the contract performance no later than the 15th day of the following month. The monthly assessment consists evaluating the principal contract support requirements by assigning numeric performance ratings for each of the CPARS performance measurement areas listed in Enclosure (3). A weighted average is then calculated for each of the performance measurements that will be used in determining the CPARS ratings.

Measure

Other

Program and Project Management Support PWS Sections 3.1, 3.2 100% inspection by COR

Project Support PWS Section 3.4 100% inspection by COR n/a

Hardware Engineering Support PWS Sections 3.5, 3.6 100% inspection by COR n/a

Software Development and Engineering Support PWS Sections 3.5, 3.7 100% inspection by COR n/a

Specialized Technical Support PWS Sections 3.5, 3.8 100% inspection by COR n/a

Information Technology Support PWS Section 3.9 100% inspection by COR n/a

Reporting PWS Section 4.0 n/a

Weighted Average†

Equivalent CPARS Rating

Weighted average values are rounded up or down to determine the CPARS standard rating.

Annual Surveillance:

The COR will perform an annual assessment of overall contract performance utilizing the CPARS evaluation rating definitions listed in Enclosure (3), and the monthly weighted average ratings. This annual assessment will occur at the end of each year following the award of the contract and cover the previous 12 months.

Measure

Quality Review of Monthly Ratings.

TBD

Schedule Review of Monthly Ratings.

Cost Control Review of Monthly Ratings.

Management Review of Quarterly Ratings.

Regulatory Compliance Review of Monthly Ratings.

Other Areas Review of Monthly Ratings.

SOURCE SELECTION INFORMATION-SEE FAR 2.101 AND 3.104

FOR OFFICIAL USE ONLY