A3.1 Attachment 3.1 (QASP).xc.pdf

General Services Administration

Federal Acquisition Service (FAS)

Greater Southwest Region

Project Number: ID07200010

Science Technology and Research Team (START2) for the

AIR FORCE MEDICAL SERVICE

ENTERPRISE-WIDE TECHNICAL

CLINICAL RESEARCH TEAM (TCRT) I

59th Medical Wing (59 MDW)

Performance Work Statement (PWS)

ATTACHMENT 3

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

Original

June 12, 2020

ID07200010

April 21, 2020

QASP

ATTACHMENT 3

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

A.1.0 Introduction This Quality Assurance Surveillance Plan (QASP) provides a systematic method to evaluate performance for the stated contract. This QASP explains the following:

● What will be monitored.

● How monitoring will take place.

● Who will conduct the monitoring.

● How monitoring efforts and results will be documented.

This QASP does not detail how the contractor accomplishes the work. Rather, the QASP is created with the premise that the contractor is responsible for management and quality control actions to meet the terms of the contract. It is the Government’s responsibility to be objective, fair, and consistent in evaluating performance.

This QASP is a “living document” and the Government may review and revise it on a regular basis. However, the Government shall coordinate changes with the contractor. Copies of the original QASP and revisions shall be provided to the contractor and Government officials implementing surveillance activities.

A.2.0 Quality Management Method

A.2.1 Quality Assurance Quality assurance activities will focus on the processes being used to manage and deliver the solution to evaluate overall project performance on a regular basis. Quality assurance will ensure the project satisfies the quality standards and will define and record quality reviews, test performance, and customer acceptance.

A.2.2 Quality Assurance Procedures

● Review and analysis of data.

● Documentation of analysis.

● Corrective action

● Documentation of lessons learned.

● Update of final quality information into official Government Past Performance databases.

A.2.3 Monitoring Processes Continuous evaluation of the vendor’s performance with respect to the effective and efficient delivery of logistical services is necessary to ensure the project is satisfying the needs of the Federal Government. Collection of data for evaluation will include those items found in Tables 1 Performance Requirement Summary.

A.2.4 In-Process Quality Monitoring The CR will use the quality tools identified in QASP to evaluate interim checkpoints of the task. These checkpoints may include, but are not limited to, the following:

● Invoices are accurate and submitted within a timely manner.

● Review of monthly status reports.

● Monthly project tracking spreadsheet (Appendix G)

● Comparison of travel cost estimates to actual costs, special emphasis on unauthorized administrative (G&A) charges.

● CTP audits.

● Contract management.

● Technical performance.

● Contract schedule.

● Approved DD250s with no exceptions.

A.2.5 Quality Control The contractor shall establish and maintain a complete Quality Control Plan (QCP) to ensure the services are performed in accordance with the PWS and commonly accepted commercial practices. Quality control activities will be performed by the vendor to verify that project management and project deliverables are of high quality and meet quality standards.

Plans typically include, but are not limited to, the following:

● A description of the inspection system, addressing all services listed in the PWS.

● Activities to be measured

● Levels of acceptable performance

● Frequency of inspections

● Title of the individual(s) who shall perform the inspections and their organizational placement.

● A description of methods for identifying, correcting and preventing defects in the quality of service performed before the level becomes unacceptable.

● The QCP will be submitted 15 days post contract award.

A.2.6 Quality Tools The client and GSA will use the following management tools to monitor the contractor’s effort:

● Monthly Status Reports

● In-Process Reviews

● Project specific reports

● Deliverables as stated in PWS

● Schedules and/or timetables

● Industry standardized processes and procedures

● Invoices

● Trip reports

● Observation by client

● Quality Standards

Quality will be measured in accordance with meeting the objectives stated in the PWS. Therefore, acceptance of deliverables and satisfactory work performance shall be based on the objectives described below:

A2.6.1 The Performance Management Feedback Loop

The performance management feedback loop begins with the communication of expected outcomes.

Performance standards are expressed in the PWS and assessed using the performance monitoring techniques outlined within the QASP.

A.2.6.2 Monitoring Forms The Government’s QA surveillance, accomplished by the COR, will be reported using the monitoring forms.

The forms, when completed, will document the Government’s assessment of the contractor’s performance under the contract to ensure that the required results are being achieved.

The COR will retain a copy of all completed QA monitoring forms.

The CO may designate a Contracting Officer Representative (COR) to perform specific administrative or technical functions. The specific rights and responsibilities of the COR for the contract shall be described in writing. A COR has no actual, apparent, or implied authority to bind the Government. CORs will be appointed in writing and the Contractor shall receive a copy of the appointment letter(s).

A.2.6.3 ANALYSIS OF QUALITY ASSURANCE MONITORING RESULTS

A.2.6.3.1 Determining Performance The Government shall use the monitoring methods cited to determine whether the performance standards, service levels, AQLs have been met. If the contractor has not met the minimum requirements, it may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels. Failure to meet the AQL may result in marginal or unacceptable performance feedback within CPARS. Likewise, if the contractor exceeds the performance standards, positive feedback may be included within CPARS.

A.2.6.3.2 Reporting On a quarterly basis, for the Base Period, the COR will prepare a written report for the GSA Project Manager and Contracting Officer summarizing the overall results of the quality assurance surveillance of the contractor’s performance. This written report, which includes the completed quality assurance monitoring form will become part of the QA documentation. It will enable the Government to demonstrate whether the contractor is meeting the stated objectives and/or performance standards, including cost/technical/scheduling objectives.

Note: The COR will prepare a written report for the GSA Project Manager and Contracting Officer summarizing the overall results of the quality assurance surveillance of the contractor’s performance every 6 months from Option Period 1 thru the end of the contract’s performance period.

A.2.6.3.3 Reviews and Resolution The COR may require the contractor’s project manager, or a designated alternate, to meet with the GSA CO and PM as deemed necessary to discuss a marginal or unacceptable performance evaluation.

The agenda of the reviews may include:

● Performance assessment data and trend analysis

● Issues and concerns of both parties

● Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis

● Recommendations for improved efficiency and/or effectiveness

● Issues arising from the performance monitoring processes.

The GSA CO must coordinate and communicate with the contractor to resolve issues and concerns regarding marginal or unacceptable performance.

The GSA CO/PM, COR, and contractor should jointly formulate tactical and long-term courses of action.

Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented.

Changes to service levels, procedures, and metrics will be incorporated as a contract modification.

Contractor Performance (Surveillance/Monitoring) Reporting Form Table 1

PERFORMANCE REQUIREMENTS SUMMARY

Required Services (Tasks)

Performance Standards

Acceptable Quality Levels

Methods of

Surveillance

1. Invoice Accuracy – invoices are submitted both accurately and on time

2. Submission of Monthly Reports (i.e.

Contract Funds Status Report, Trip Report, Contractor Employee Listing, Monthly Status Reports, Progress Reports, etc.)

3. Accuracy of Monthly Project Tracking Spreadsheet

4. CPARS Criteria

● Schedule

● Quality (Services & Deliverables)

● Cost

No more than 2 errors during the reporting period

No more than 2 errors per month

No errors period for each criteria

100%

File reviews, periodic inspections, and random, observations, customer complaint

QUALITY ASSURANCE MONITORING FORM

SURVEY PERIOD: _______________________________________________

LEVEL OF SURVEILLANCE (Check):

❒ Monthly ❒ Quarterly ❒ Semi-Annual (every 6 months)

Required Services (Tasks)

Performance Standards

Acceptable Quality Levels

Method of

Surveillance Used

Performance Results

Invoice Accuracy – invoices are submitted both accurately and on time

Submission of Monthly Reports

Accuracy of Monthly Project Tracking Spreadsheet PWS Attachment 1

CPARS Criteria

● Schedule

● Quality (Services &

No more than 2 errors period

No more than 2 errors per month

No errors

❒ Random

Sampling ❒ 100%

Inspection ❒ Periodic

Inspection ❒ Customer

Complaint

Sampling ❒ 100%

Inspection ❒ Periodic

Inspection

❒ 100%

Inspection ❒ Periodic

Inspection

Schedule

❒ Meets

Standards ❒ Does Not

Meet Standards

❒ Meets

Standards

Schedule

Deliverables)

● Cost period for each criteria

❒ 100%

Inspection ❒ Periodic

Inspection

Quality

Sampling ❒ 100%

Inspection ❒ Periodic

Inspection

Cost

Sampling ❒ 100%

Inspection ❒ Periodic

Inspection

Quality

Standards

Cost

Standards

Narrative of Performance During Survey Period: _____________________________

PREPARED BY: ___________________________________ DATE: _________________

Contractor Performance Assessment Report (CPARS)

General Performance Rating

Summarize contractor performance and check the rating for each category (See attached Rating

Guidelines).

a. QUALITY OF PRODUCT OR SERVICE Unsatisfactory Marginal Satisfactory Very Good Exceptional N/A

Comments:

b. SCHEDULE Unsatisfactory Marginal Satisfactory Very Good Exceptional N/A

c. COST CONTROL Unsatisfactory Marginal Satisfactory Very Good Exceptional N/A

d. MANAGEMENT Unsatisfactory Marginal Satisfactory Very Good Exceptional N/A

e. UTILIZATION OF SMALL BUSINESS Unsatisfactory Marginal Satisfactory Very Good Exceptional N/A

f. REGULATORY COMPLIANCE Unsatisfactory Marginal Satisfactory Very Good Exceptional N/A

g. OTHER AREAS Unsatisfactory Marginal Satisfactory Very Good Exceptional N/A

Subcontracts

Are subcontracts involved? __Yes __No (Check one)

Government Comment on subcontracts (2000 characters maximum):

Given what I know about the Contractor’s ability to execute what they promise in their proposal, I would award to them today given I had a choice? (Choose one of the following responses)

Definitely would not

Probably would not

Might or might not

Probably would

Definitely would

Have all deliverables been reviewed and accepted? __Yes __No (Check one)

Rating Guidelines

Contractor Evaluation Category Definition

● Quality of Product or Service. Assess the Contractor's conformance to contract/order requirements, specifications and standards of good workmanship (e.g., commonly accepted technical, professional, environmental, or safety and health standards). MANDATORY

For example: Are reports/data accurate? Does the product or service provided meet the specifications of the contract/order? Does the Contractor's work measure up to commonly accepted technical or professional standards? Assess the degree of Government technical direction required to solve problems that arise during performance.

For Operations Support: Assess how successfully the Contractor meets program quality objectives such as ability to produce, reliability, maintainability and ability to inspect. The Assessing

Official must be flexible in how Contractor success is measured; e.g., using data from field reliability and maintainability and failure reports, user comments and acceptance rates, and scrap and rework rates. These quantitative indicators may be useful later, for example, in source selection evaluations, in demonstrating continuous improvement, quality and reliability leadership that reflects progress in total quality management. Assess the Contractor's control of the overall production process to include material control, shop planning and control, and status.

● Schedule. Assess the timeliness of the Contractor against the completion of the contract, task orders, milestones, delivery schedules, and administrative requirements (e.g., efforts that contribute to or effect the schedule variance). MANDATORY

This evaluation of the Contractor's adherence to the required delivery schedule should include the

Contractor's efforts during the evaluation period that contributes to or effects the schedule variance. This element applies to contract/order closeout activities as well as contract performance.

Instances of adverse actions such as the assessment of liquidated damages, or issuance of Cure

Notices, Show Cause Notices, and Delinquency Notices are indicators of problems which may have resulted in variance to the contract/order schedule and should therefore be noted in the evaluation.

● Cost Control (Not required for Firm Fixed Price or Firm Fixed Price with Economic Price

Adjustment). Assess the Contractor's effectiveness in forecasting, managing, and controlling contract/order cost. MANDATORY

For example, does the Contractor keep within the total estimated cost (what is the relationship of the negotiated costs and budgeted costs to actuals)? Did the Contractor do anything innovative that resulted in cost savings? Were billings current, accurate and complete? Are the Contractor's budgetary internal controls adequate?

Evaluation information regarding performance under an Undefinitized Contract Action (UCA), including letter contracts and actions where price has not yet been negotiated, shall be included in the annual evaluation. If the final negotiated contract/order type is not a cost-type, cost information for the period the

UCA was in effect shall be included under the cost element. If the final negotiated contract/order type is a cost-type, cost information for the entire period of performance shall be included under the Cost rating element. The narrative shall fully explain the Contractor's performance during the UCA, including definitization of the contract/order. The Contractor's performance under the UCA shall be separately identified but considered in the overall annual ratings.

● Management Assess the integration and coordination of all activity needed to execute the contract or order, specifically the timeliness, completeness and quality of problem identification, corrective action plans, proposal submittals, the Contractor’s history of reasonable and cooperative behavior (to include timely identification of issues in controversy), customer satisfaction, timely award and management of subcontracts. Include, as applicable, information on the following:

- Is the Contractor oriented toward the customer?

- Is interaction between the Contractor and the Government satisfactory, or does it need improvement?

- Include the adequacy of the Contractor’s accounting, billing, and estimating systems and the Contractor’s management of Government Furnished Property (GFP) if a substantial amount of GFP has been provided to the Contractor under the contract or order.

- Address the timeliness of awards to subcontractors and management of subcontractors, including subcontract costs. Consider efforts taken to ensure early identification of subcontract problems, and the timely application of corporate resources to preclude subcontract problems from impacting overall prime contractor performance.

- Assess the Prime Contractor’s effort devoted to managing subcontracts and whether subcontractors were an integral part of the Contractor’s team. Are the Contractor’s management, on-site, and home office personnel exhibiting the capacity to adequately plan, schedule, resource, organize, and otherwise manage the work? If not, describe and relate to other rated elements. MANDATORY

● For example, Management Responsiveness. Assess the timeliness, completeness, and quality of problem identification, corrective action plans, proposal submittals (especially responses to change orders, Engineering Change Proposals (ECPs), or other UCAs), the Contractor's history of reasonable and cooperative behavior, effective business relations, and customer satisfaction.

Consider the Contractor’s responsiveness to the program as it relates to meeting contract or order requirements during the period covered by the report.

Utilization of Small Business. This area must be rated for all contracts and task orders that contain a

Small Business subcontracting goal. Assess whether the Contractor provided maximum practicable opportunity for Small Businesses (including Alaska Native Corporations (ANCs) and Indian Tribes)

(including Small Disadvantaged Businesses (which also includes ANCs and Indian Tribes), Women-Owned

Small Businesses, Economically Disadvantaged Women-Owned Small Businesses, HUBZones, Veteran-

Owned Small Businesses, Service Disabled Veteran-Owned Small Businesses, and ANCs and Indian

Tribes that are not Small Disadvantaged Businesses or Small Businesses) to participate in contract/order performance consistent with efficient performance of the contract/order.

Assess compliance with all terms and conditions in the contract/order relating to Small Business participation (including FAR 52.219-8, Utilization of Small Businesses and FAR 52.219-9, Small Business

Subcontracting Plan (when required)). Assess any Small Business participation goals, to include AbilityOne goals, which are stated separately in the contract/order. Assess achievement on each individual goal stated within the contract/order or subcontracting plan including good faith effort if the goal was not achieved.

It may be necessary to seek input from the Small Business Specialist or Contracting Officer in regards to the Contractor's compliance with these criteria. For DoD, in cases where the Contractor has a comprehensive subcontracting plan, request the DCMA Comprehensive Subcontracting Plan Manager to provide input including any program specific performance information.

For contracts/orders subject to a commercial subcontracting plan, the Utilization of Small Business factor should be rated Satisfactory as long as an approved plan remains in place, unless liquidated damages have been assessed by the Contracting Officer who approved the commercial plan (see FAR 19.705-7(h)).

In such case, the Utilization of Small Business area must be rated Unsatisfactory.

Subcontracting Plan

A subcontracting plan is required in accordance with FAR 19.7 and the applicable clauses incorporated into this contract.

Regulatory Assess compliance with all terms and conditions in the contract or order relating to applicable regulations and codes. Consider aspects of performance such as compliance with financial, environmental

(example: Clean Air Act, Clean Water Act), safety, and labor regulations, as well as any other reporting requirements in the contract. Consider questions such as the following:

- Has the Contractor complied with all contract clauses and reporting requirements (e.g., FAPIIS, FAR

Subsection 9.104-6, FAR Clause 52.209-9)?

- Has the Contractor complied with Cost Accounting Standards and disclosure statements (FAR Subpart

30.2, FAR Clauses 52.230-2, 52.230-3, 52.230-4, 52.230-5, 52.230-6)?

- Has the Contractor complied with the reporting requirements relating to recovered material content utilized in contract performance (see FAR Subpart 23.4)?

- Has the Contractor complied with contractual safety requirements and labor laws (see FAR Subsection

22.407, FAR Clauses 52.222-4, 52.222-34, 52.222-20)?

- Perform engineering surveillance to assess compliance with contractual terms for schedule, cost, and technical performance in the areas of design, development, and production (FAR Subpart 42.302).

- Has the Contractor complied with Hazardous Material Identification and Material Safety Data (see FAR

Subpart 23.3, FAR Clauses 52.223-3)?

- Has the Contractor complied with specifications or other contractual requirements requiring the delivery or use of environmentally preferable products, energy-efficient products, products containing recovered materials, and biobased products (FAR Subparts 23.2, 23.4, 23.7)? Monitor the Contractor's environmental practices for adverse impact on contract performance or contract cost, and for compliance with environmental requirements specified in the contract.

- Has the Contractor implemented an effective safety program; one which minimizes/mitigates potential accidents (FAR Subsection 36.513, FAR Clause 52.236-13)?

- Is the Contractor complying with affirmative action and EEO compliance requirements (see FAR Subpart

22.8, FAR Clauses 52.222-26, 52.222-27)?

- Has the Contractor complied with combating trafficking in persons (see FAR Subpart 22.17, FAR Clause

52.222-50)?

Ensuring sustainable acquisition compliance in federal contracts requires the evaluation of contractor performance inclusive of all requirements from the contract SOW/PWS; contract clauses; and other applicable contract terms and conditions. The GSA Sustainable Facilities Tool is a one-stop online resource that provides useful information regarding sustainability regulations and guidelines. Additionally, the information contained in the chart below regarding the federal green programs and the applicable legal basis/regulation may be useful to the Assessing Official in determining how to evaluate the Contractor’s performance in these areas.

For example, if evaluating contractor performance on a construction contract to retrofit plumbing, the evaluator should check the SOW, the Sustainable Facilities Tool, and the chart below. The SOW most likely specified WaterSense products. As such, the evaluator should check the WaterSense information from the Sustainable Facilities Tool and the chart below to ensure that the various products used in support of the contract meet the WaterSense criteria.

● Other Areas. Specify additional evaluation areas that are unique to the contract/order, or that cannot be captured elsewhere on the form. More than one type of entry may be included, but should be separately labeled.

Contractor Evaluation Ratings Definition

• Exceptional- Performance meets contractual requirements and exceeds many to the Government’s benefit. The contractual performance of the element or sub-element being assessed was accomplished with few minor problems for which corrective actions taken by the contractor was highly effective.

Note: To justify an Exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an

Exceptional rating. Also, there should have been NO significant weaknesses identified.

• Very Good- Performance meets contractual requirements and exceeds some to the Government’s benefit. The contractual performance of the element or sub-element being assessed was accomplished with some minor problems for which corrective actions taken by the contractor was effective.

Note: To justify a Very Good rating, identify a significant event and state how it was a benefit to the

Government. There should have been no significant weaknesses identified.

• Satisfactory- Performance meets contractual requirements. The contractual performance of the element or sub-element contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory.

Note: To justify a Satisfactory rating, there should have been only minor problems, or major problems the contractor recovered from without impact to the contract. There should have been NO significant weaknesses identified. Per DOD policy, a fundamental principle of assigning ratings is that contractors will not be assessed a rating lower than Satisfactory solely for not performing beyond the requirements of the contract.

• Marginal- Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being assessed reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented.

Note: To justify Marginal performance, identify a significant event in each category that the contractor had trouble overcoming and state how it impacted the Government. A Marginal rating should be supported by referencing the management tool that notified the contractor of the contractual deficiency (e.g., management, quality, safety, or environmental deficiency report or letter).

• Unsatisfactory- Performance does not meet most contractual requirements and recovery is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective.

Note: To justify an Unsatisfactory rating, identify multiple significant events in each category that the

Contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone constitutes an unsatisfactory rating. An

Unsatisfactory rating should be supported by referencing the management tools used to notify the contractor of the contractual deficiencies (e.g., management, quality, safety, or environmental deficiency reports, or letters).

NOTE 1: Plus or minus signs may be used to indicate an improving (+) or worsening (-) trend insufficient to change the assessment status.

NOTE 2: N/A (not applicable) should be used if the ratings are not going to be applied to a particular area for evaluation.

[END OF ATTACHMENT 3]