B.08.02.03 QASP_CCPL118_ENV-22-0051.pdf

Quality Assurance Surveillance Plan FY21 Remedial Action Operation – CCVI018

U.S. Army Garrison (USAG) Italy

Prepared By

U.S. Army Corps of Engineers, Europe District

Environmental Division Konrad Adenauer Ring 39

65187 Germany

June 2022

LIST OF TABLES

Table 1 – Subject Matter Experts Table 2 – Evaluation Areas & Ratings Criteria Table 3 – Key Milestones/Deliverables by Evaluation Areas Table 4 – Specific Evaluation Criteria for Established Key Milestones/Deliverables

LIST OF ATTACHMENTS

Attachment A: Quality Assurance Monitoring Form Attachment B: Corrective Action Form

LIST OF ACRONYMS & ABBREVIATIONS

COR Contracting Officer’s Representative KO Contracting Officer PM Project Manager PBA Performance Based Acquisition PMP Project Management Plan POC Point of Contact PWS Performance Work Statement QAMF Quality Assurance Monitoring Form QAR Quality Assurance Report QASP Quality Assurance Surveillance Plan SME Subject Matter Experts USACE U.S. Army Corps of Engineers

1.0 OVERVIEW

1.1 Introduction. This Quality Assurance Surveillance Plan (QASP) sets forth the procedures and guidance that the Contracting Officer’s Representative (COR) will use in evaluating the technical and quality performance of the Contractor in accordance with the terms and conditions of the performance work statement (PWS) for Contract No. TBD TO: TBD. A copy of the signed final plan will be furnished to the Contractor so that the Contractor will be aware of the methods that the COR will use in evaluating performance of this contract.

1.2 Purpose. The purpose of the QASP is to ensure that the performance of specific activities and the completion of project milestones are accomplished in accordance with all requirements set forth in the PWS and outlined in the Project Management Plan (PMP). This QASP describes the mechanism for documenting noteworthy accomplishments or discrepancies for work performed by the Contractor.

Information generated from COR’s surveillance activities will directly feed into performance discussions with the Contractor. The intent is to ensure that the Contractor performs in accordance with performance metrics set forth in the contract documents, the Army receives the quality of services called for in the contract, and the Army only pays for acceptable services received.

The QASP is intended to accomplish the following:

• Define the role and responsibilities of participating Army officials.

• Define the key milestones, deliverables, and standards that will be assessed.

• Describe the surveillance methodology that will be employed by the Army in assessing the

Contractor’s performance.

• Describe the surveillance documentation process and provide copies of the forms that the Army will use in evaluating the Contractor’s performance.

• Outline quality assurance procedures to be employed by the Government during performance of this task order.

• Define Evaluation Areas, including Quality, Schedule, Management, Regulatory Compliance

(with Safety).

• Define Exceptional, Very Good, Satisfactory, Marginal, and Unsatisfactory performance standards for key milestones, deliverables, and standards.

• Ensure clarity on recent changes to the formulation of contractor evaluations, from an overall/cumulative rating to a recommend/do not recommend.

• Outline corrective action procedures.

• Describe payment procedures.

2.0 ROLES AND RESPONSIBILITIES OF QUALITY ASSURANCE ARMY OFFICIALS

2.1 Contracting Officer. The Contracting Officer (KO) has overall responsibility for overseeing the Contractor’s performance. The KO is responsible for the day-to-day monitoring of the Contractor’s performance in the areas of contract compliance, and contract administration; reviewing the COR’s assessment of the Contractor’s performance; and resolving all differences between the COR’s assessment and the Contractor’s assessment of performance. It is the KO that assures the Contractor receives impartial, fair, and equitable treatment under the contract. The KO is ultimately responsible for the final determination of the adequacy of the Contractor’s performance. The KO for this contract is Dayon Santos at the U.S. Army Corps of Engineers (USACE), Europe District, 0611-9744-2427.

2.2 Contracting Officer Representative (COR). The Contracting Officer’s Representative (COR) is responsible for technical administration of the project and assures proper Army surveillance of the Contractor’s performance. The COR is responsible for monitoring, assessing, recording, and reporting on the technical performance of the Contractor on a day-to-day basis. The COR and Project Manager (PM) for this contract is Mary Mason at the U.S. Army Corps of Engineers (USACE), Europe District, 0611- 9744-2651, Mary.C.Mason@usace.army.mil.

mailto:Mary.C.Mason@usace.army.mil

2.3 Technical Expertise and Subject Matter Experts. The KO and COR may call upon the technical expertise of other Army Officials and subject matter experts (SME) as required. These Army Officials and SMEs may be called upon to review technical documents and products generated by the Contractor.

For this contract, the following Army Officials and SMEs have been identified:

TABLE 1 – Subject Matter Experts

U S Corps of Engineers, Europe District Mary Mason CC Program Manager, USACE

U.S. Army Garrison (USAG) Italy Alberto Cristoferi Environmental Engineer, DPW-ED

If additional Army Officials and SMEs are identified as work progresses, the QASP will be modified to capture this information.

3.0 METHODS FOR PERFORMANCE ASSESSMENT

3.1 Evaluation Areas. The Contractor’s performance will be evaluated by assessing the key milestones and deliverables above according to the standards of Quality, Schedule, Management, and Regulatory Compliance. Other Evaluation Areas can be included at the discretion of the COR and Project Manager (PM) and will be provided to the Contractor in a revised QASP prior to its finalization. Each Evaluation Area is rated as Exceptional, Very Good, Satisfactory, Marginal, or Unsatisfactory.

3.2 Evaluation Areas and Ratings Criteria.

TABLE 2 – Evaluation Areas & Ratings Criteria

EVALUATION AREAS & EXAMPLE INDICATORS

Quality • Product performance relative to contract’s performance parameters

• Performance in terms of contract’s quality objectives

• Use quantitative indicators wherever possible

• Contractor’s management of the quality control program

• Quality of the work or service

Schedule • Timeliness of the Contractor against the completion of the contract, TOs, milestones, delivery schedules, and administrative requirements

• Contractor’s adherence to the required delivery schedule, that contributes to or affects the scheduled variance

• Adverse actions such as the assessment of liquidated damages or issuance of Cure Notices, Show Cause, and Delinquency Notices are indicators of problems which may have resulted in variance to the contract schedule and should, therefore, be noted in the evaluation

• Adequacy of Initial Progress Schedule

• Submission of Required Documentation

• Adherence to Approved Schedule

• Submission of Updated & Revised Progress Schedules

• Resolution of Delays

Management • Integration and coordination of activity

• Problem identification

• Corrective action plans

• Reasonable and cooperative behavior

• Customer satisfaction

• Subcontract management

• Program management

• Management of key personnel

Regulatory Compliance with U.S. and Host Nation regulations

• Compliance with regulations and codes

• Financial

• Environmental

• Labor o Correction of Noted Deficiencies o Compliance with Labor Laws and Regulations with Specific

Attention to the Davis-Bacon Act and o EEO Requirements o Payrolls Properly Completed and Submitted

• Safety o Adequacy of Safety Plan o Correction of Noted Deficiencies o Implementation of Safety Plan

• Other Reporting requirements in the contract

EVALUATION AREA RATINGS DEFINITIONS*

Exceptional Performance meets contractual requirements and exceeds many to the

Government’s benefit.

The contractual performance of the element or sub-element being assessed was accomplished with few minor problems for which corrective actions taken by the contractor was highly effective. To justify an exceptional rating, identify multiple significant events and state how they were of benefit to the Government. A singular benefit, however, could be of such magnitude that it alone constitutes an exceptional rating. Also, there should have been NO significant weaknesses identified.

Very Good Performance meets contractual requirements and exceeds some to the Government’s benefit.

The contractual performance of the element or sub-element being assessed was accomplished with some minor problems for which corrective actions taken by the contractor was effective. To justify a very good rating, identify a significant event and state how it was a benefit to the Government. There should have been NO significant weaknesses identified.

Satisfactory Performance meets contractual requirements. The contractual performance of the element or sub-element being assessed contains some minor problems for which corrective actions taken by the contractor appear or were satisfactory. To justify a satisfactory rating, there should have been only minor problems, or major problems the contractor recovered from without impact to the contract.

There should have been NO significant weaknesses identified. Per DOD Policy, a fundamental principle assigning ratings is that contractors will not be assessed a rating lower than satisfactory solely for not performing beyond the requirements of the contract.

Marginal Performance does not meet some contractual requirements. The contractual performance of the element or sub-element being assessed reflects a serious problem for which the contractor has not yet identified corrective actions. The contractor’s proposed actions appear only marginally effective or were not fully implemented. To justify marginal performance, identify a significant event in each category that the contractor had trouble overcoming and state how it impacted the Government. A marginal rating should be supported by referencing the management tool that notified the contractor of the contractual deficiency (e.g., management, quality, safety or environmental deficiency report or letter).

Unsatisfactory Performance does not meet most contractual requirements and recover is not likely in a timely manner. The contractual performance of the element or sub-element contains a serious problem(s) for which the contractor’s corrective actions appear or were ineffective. To justify an unsatisfactory rating, identify multiple significant events in each category that the contractor had trouble overcoming and state how it impacted the Government. A singular problem, however, could be of such serious magnitude that it alone would constitute an unsatisfactory rating. An unsatisfactory rating should be supported by referencing the management tools used to notify the contractor of the contractual deficiencies (e.g., management, quality, safety or environmental deficiency report or letter).

*The guidelines provided for issuing ratings are subjective in nature, and these ratings will be supported by the weight of evidence documented during the government's surveillance efforts.

3.3 Performance Assessment Process. If a deliverable is rated as being unsatisfactory for Quality or Regulatory Compliance at the time that the approved PWS deadline for the milestone expires, the Contractor may receive an unsatisfactory rating for Schedule, unless there is an Army approved delay that extends the PWS deliverable/milestone. This will only take place during the rating period and will not necessarily influence future/later ratings.

3.3.1 Army Approved Delays. At the discretion of the COR, the performance standard of Schedule outlined in Table 4, may not be monitored due to Army-Approved delays. QAMFs will track and report this based on Contractor monthly reports to the COR.

3.4 Recommendation. In lieu of an overall rating, the Contractor will either be recommended/not recommended for similar future requirements.

3.5 Key Milestones/Deliverables to be assessed. The following milestones and associated deliverables will be evaluated in accordance with this QASP:

3.5.1 Key Milestones

• Kick-Off meeting Minutes

• Health and Safety Plan (HASP)

• COR acceptance of the Final Work Plan and PMP

• Site Investigation Report

• Host Nation Coordination

3.5.2 Key Deliverables

• Task 1: Work Plan (WP), Project Management Plan (PMP), Health and Safety Plan (HASP)

• Task 2: Site Investigation Report

3.5.3 Monitoring Method

• 100% Inspection: All project milestones and deliverables will be evaluated through 100% inspection by onsite inspection or document review. The USACE Project Manager will document performance for each completed milestone or deliverable prior to payment, as described in Section 5.0.

• Periodic Inspection: At the USACE Project Manager’s discretion, periodic inspections will be conducted to evaluate progress toward key milestones and deliverables. This will include QA Inspections (safety, geophysics, chemical data packages, etc.) by a government representative.

The USACE Project Manager may also complete a periodic progress inspection if he/she believes that deficiencies exist that must be addressed prior to milestone or deliverable completion. While corrective action or re-performance will be required, if necessary, the Contractor will not be financially penalized for unacceptable performance recorded in periodic progress reports, provided that final performance evaluation of the milestone or deliverable is deemed acceptable.

• Customer Feedback: Contractor performance feedback will be obtained through periodic inquiries by the USACE Project Manager with project stakeholders. The purpose of these inquiries would be to supplement the other forms of evaluation and to also provide the Contractor with constructive criticism and/or recognition for the project deliverables or milestones completed. Customer feedback received will be thoroughly validated to ensure it relates to the requirements of the PWS and will be used in a prudent manner by the COR. Customer feedback will also be solicited in the form of a concurrence letter by the Contractor from appropriate stakeholders for key deliverables.

3.5.4 Additional Surveillance Activities

Additional Government surveillance activities may include, but are not limited to, the following:

• Review and approval of meeting minutes from Kickoff Meetings

• Review of data deliverables

• Oversight of field work activities

• Review of uploaded electronic deliverables

• Review of the Contractor’s quality control documentation.

• Review of the Contractor’s safety records

Since this is a PBA, it is possible to have a twelve-month or longer period where no milestones/deliverables are met. When this is the case, the quality of Contractor performance will be evaluated using additional surveillance activities, including but not limited to, Contractor monthly reports, government oversight, and review of documentation.

3.5.5 Indicators and Documentation

• COR acceptance

• Meeting minutes

• Memos

• Worksheets

• Correspondence (letters, email)

Table 3 – Key Milestones/Deliverables by Evaluation Areas

Milestone/Deliverable* Quality Schedule Management Regulatory Compliance (incl. Safety)

Draft/Draft Final Updated Project Management Plan, Work Plan and Health and Safety Plan

X X

Final Work Plan and HASP X X X X Kick off Meeting Minutes X X Field Work X X X Draft/Draft Final Site Investigation Report

X X

Final Site Investigation Report X X X X

* Includes Key Milestones and Key Deliverables from PWS.

3.6 Performance Standards. The Contractor’s performance will be evaluated by assessing the key milestones and deliverables above according to the standards of Quality, Schedule, Management, Regulatory Compliance (which includes Safety). In addition, the Contractor’s performance will be evaluated for the standard of Safety during any fieldwork. For each of these performance standards, the

COR will assign one of five ratings of the Contractor’s performance: Exceptional, Very Good, Satisfactory, Marginal, or Unsatisfactory, as shown in Table 4.

Table 4 – Specific Evaluation Criteria for Established Key Milestones/Deliverables

Performance Standard

Exceptional Very Good Satisfactory Marginal Unsatisfactory

Quality Draft, Draft Final, and Final deliverables are of excellent quality, approved as submitted, or with no substantive comments limited to grammar, spelling, or terminology.

Draft deliverables are of high quality and comments are mostly minor. Final deliverables are approved after one (1) round of comments on the Draft through acceptance of response to comments table and back check of Final report against original comments. No further revisions are required.

Draft deliverables are of acceptable quality with only a few numbers of comments identifying major weaknesses.

Final deliverables are approved after two (2) rounds of comments on Draft. No further revisions are required.

Draft deliverables are of poor quality with a significant number of comments identifying major weaknesses or deficiencies.

Final deliverables require more than two (2) rounds of comments on Draft before being approved.

(e.g., changes are required to the Final document due to inadequate incorporation of comments).

Draft deliverables are of very poor quality and are rejected for resubmittal without comment. Final deliverables did not comply with contract requirements, or one or more document versions required more than three (3) rounds of comments before being approved.

Army audit finds that the data collect and/or the work performed exceeds the requirement of the SOW. No deficiencies noted

Army audit of work does not identify any deficiencies that compromise the quality of the data collected or work performed.

Army audit of work identifies deficiencies that do not compromise the quality of the data collected or work performed and can be corrected.

Army audit of work identifies deficiencies that compromise the quality of the data collected or work performed but were corrected.

Army audit of work identifies deficiencies that compromise the quality of the data collected or work performed and cannot be corrected.

Schedule Contractor Achieves milestone more than 60 days ahead of schedule, per criteria established in the SOW and this QASP.

Contractor Achieves milestone less than 60 days but more than 30 days ahead of schedule, per criteria established in the SOW and this

QASP.

Contractor achieves milestone according to the schedule, per criteria established in the SOW and this

QASP.

Contractor achieves milestone more than 30 days but less than 60 days behind schedule (unless the COR notes Army delay), per criteria established in the SOW and this

QASP.

Contractor achieves milestone more than 60 days behind schedule (unless the COR notes Army delay), per criteria established in the SOW and this QASP.

For KOM mins, PMP/WP/HASP excellent rating is achievement of milestone 14

For KOM mins, PMP/WP/HASP very good rating is achievement of milestone 7

For KOM mins, PMP/WP/HASP satisfactory rating is achievement of

For KOM mins, PMP/WP/HASP marginal rating is achievement of milestone 7

For KOM mins, PMP/WP/HASP unsatisfactory rating is achievement of

Performance Standard

Exceptional Very Good Satisfactory Marginal Unsatisfactory days ahead of schedule.

days ahead of schedule.

milestone on schedule.

days behind schedule.

milestone 14 days behind schedule.

Management All personnel proposed by the contractor were assigned to the project.

All personnel proposed by the contractor were assigned to the project. Some personnel were substituted by higher qualified individuals.

All personnel proposed by the contractor were assigned to the project. Some personnel were substituted by equally qualified individuals.

All personnel proposed by the contractor were assigned to the project. Most personnel were substituted by equally qualified individuals.

All personnel proposed by the contractor were assigned to the project. Some personnel were substituted by lesser qualified individuals.

Informal poor performance feedback on conduct of personnel is provided by the COR but are corrected.

Formal letter of poor performance feedback on conduct of personnel is provided by the COR but are corrected.

Written request from USACE requesting removal of assigned personnel for poor performance or notification of poor performance is provided by the COR and is not corrected.

Zero (0) instances of resource management issues creating a negative impact to the activity.

No more than one (1) instance of resource management issues creating a negative impact to the activity.

No more than two (2) instances of resource management issues creating a negative impact to the activity.

No more than three (3) instances of resource management issues creating a negative impact to the activity.

More than three

(3) instances of resource management issues creating a negative impact to the activity.

Regulatory Compliance (includes Safety)

Contractor obtains concurrence on deliverables from the PDT.

This concurrence is obtained independently with little to no involvement and coordination required by the Government.

Contractor obtains concurrence on deliverables from the PDT.

This concurrence is obtained independently with limited involvement and coordination required by the Government.

Contractor obtains concurrence on deliverables from the PDT.

This concurrence is obtained with a moderate level of involvement and coordination required by the Government.

Contractor obtains concurrence on deliverables from the PDT.

This concurrence is obtained with a significant level of involvement and coordination required by the Government.

Contractor does not obtain concurrence on deliverables from the PDT.

No significant safety deficiencies are reported during QA inspection of fieldwork.

No lost time accidents or injuries are

No more than one (1) serious safety deficiencies are reported during QA inspection of fieldwork. If any serious safety deficiency

No more than two (2) serious safety deficiencies are reported during QA inspection of fieldwork. If any serious safety deficiency

No more than three (3) serious safety deficiencies are reported during QA inspection of fieldwork. If any serious safety deficiency is

More than three

(3) serious safety deficiencies are reported during QA inspection of field activities, or a serious safety deficiency is reported but

Performance Standard

Exceptional Very Good Satisfactory Marginal Unsatisfactory recorded during the fieldwork.

is noted during the project, appropriate investigation, corrective action, implementation, and written verification of the corrective action are provided to the Army. No lost time accidents or injuries are recorded during the fieldwork.

is noted during the project, appropriate investigation, corrective action, implementation, and written verification of the corrective action are provided to the Army. No lost time accidents or injuries are recorded during the fieldwork.

noted during the project, appropriate investigation, corrective action, implementation, and written verification of the corrective action are provided to the Army. No more than one lost time accident or injury is recorded during the fieldwork.

not properly investigated and corrected, or two or more lost time accidents or injuries is recorded during field activities

4.0 SURVEILLANCE DOCUMENTATION

4.1 Quality Assurance Monitoring Form. The COR or designee will use the Quality Assurance Monitoring Form (QAMF) (Attachment A) to record evaluation of the Contractor’s performance for each payment milestone or final deliverable in accordance with the methodology described in Section 3.0 and Section 4.0. The USACE Project Manager must substantiate, through narratives on the form, all Exceptional and Unacceptable ratings. Performance at the acceptable level is expected from the Contractor. At a minimum, the evaluation form will indicate actual and scheduled delivery times and number of reviews required to achieve the final product.

The USACE Project Manager will forward copies of all completed QAMFs to the USACE COR within 7 days of performing the inspection. The USACE Project Manager will forward all completed quality assurance monitoring forms to the U.S. Army Garrison (USAG) Italy and Contractor within 14 days.

4.2 Corrective Action Process. When a key milestone/deliverable receives a marginal or unacceptable rating, the Contractor will explain, within 15 days, in writing to both the USACE COR and USACE Project Manager why performance was marginal or unacceptable, how performance will be returned to acceptable levels, and how recurrence of the problem will be prevented in the future. The Contractor shall use the corrective action form as part of this process (Attachment B). The USACE COR will review the proposed corrective action with the U.S. Army Garrison (USAG) Italy and USACE Project Manager, and U.S. Army Garrison (USAG) Italy POC, as necessary, to determine if it will be accepted.

4.3 KO and COR Roles in Surveillance Process. The USACE Project Manager will provide the COR and KO with copies of all completed QAMFs. When appropriate, the COR and/or KO may investigate further to determine if all the facts and circumstances surrounding the event were considered in the USACE Project Manager opinions outlined on the form. The COR and/or KO will immediately discuss any unacceptable rating with the Contractor’s Program Manager to assure that corrective action is promptly initiated. At the end of the contract performance period, the USACE Project Manager will prepare a written report for the COR and KO summarizing the overall results of the surveillance of the Contractor’s performance during the contract. This report will become part of the formal QA documentation. The USACE Project Manager will maintain a complete QA file. This file will contain copies of all performance evaluation forms and any other related documentation. The USACE Project Manager will forward these records through the COR and to the KO at termination or completion of the contract.

4.4 Technical Quality Assurance Monitoring. In general, all work will be evaluated in terms of how well the requirements of the task order are satisfied, the extent to which the work performed follows the approach found in the contractor’s technical proposal and/or implements the decision of Technical Project Planning, and clarity of documentation. At the discretion of the COR or the Contracting Officer or Specialist, other government officials approved by the Contracting Officer or Specialist may be asked to evaluate a particular deliverable or set of deliverables. The results of all Technical Quality Assurance Monitoring will be documented using a Technical Review Form. Technical Quality Assurance Monitoring Documentation will document technical criteria evaluated. Relevant forms are included in Attachment A, B and will be updated as needed.

5.0 PAYMENT

5.1 Acceptable Performance. The Contractor will also be required to perform a milestone presentation per the PWS. At the discretion of the COR, these milestone presentations may be conducted as part of the next regularly scheduled Project Meeting. Full payment for a milestone will be provided upon verification of overall acceptable performance as indicated on the QAMF. The contractor should provide an invoice to the USACE Project Manager after receipt of the QAMF from the USACE indicating acceptable performance. If a QAMF is not provided to the Contractor within 14 days of completion of the milestone the Contractor will submit an invoice.

5.2 Unsatisfactory Performance. If a milestone or deliverable receives an unsatisfactory rating for either the Quality or Regulatory Compliance performance standard, re-performance is required until the deliverable receives an acceptable rating. This re-performance is required regardless of cost or schedule constraints that may result from the unsatisfactory performance, unless the KO has opted to terminate the contract.

QASP Approval:

Mary C. Mason Date:

Contracting Officer’s Representative

ATTACHMENT A: QUALITY ASSURANCE MONITORING FORM

W912GB-XX-P-00XX

Date: Date

Customer: USAG

Milestone, Deliverable &/or Standard: Deliverable Name

Survey Period: Date(s)

Method of Surveillance:

Prepared By: Sal Van Wert, 314-570-2404

Narrative Discussion of Contractor Performance during Survey Period:

Corrective Action Required: Yes_______ No _______

I certify that I have checked the quantities covered by this bill or estimate: that the work was actually performed; that the quantities are correct and consistent with all previous computations as actually checked; that the quantities and amounts are wholly consistent with the requirements of the contract or other instrument involved.

Name Date Title

ATTACHMENT B: CORRECTIVE ACTION FORM

W912GB-XX-P-00XX

1) Date:

2) Customer:

3) Milestone, Deliverable &/or Standard:

4) Survey Period:

5) Author’s Name and Phone Number:

6) Description of the Failure/Deficiency that Precipitated the Corrective Action:

7) Description of the Criterion that the Failure/Deficiency was Evaluated Against:

8) Personnel Involved in Identification of the Failure/Deficiency, Determination of the Appropriate Corrective Action, Approval of the Corrective Action, and Implementation of the Corrective Action:

9) Description of the Corrective Action that was Required:

10) Date/Time of Implementation of the Corrective Action:

11) Follow Up Information to Prevent Recurrence of Failure/ Deficiency (i.e., Need For Revision of Procedures or Specifications):

12) Personnel Responsible for Follow-Up Work:

13) Planned Date for Follow Up Surveillance:

14) Other Notes:

1.2 Purpose. The purpose of the QASP is to ensure that the performance of specific activities and the completion of project milestones are accomplished in accordance with all requirements set forth in the PWS and outlined in the Project Management Pl...