Exhibit E.1 - Quality Assurance Surveillance Plan.pdf

QUALITY ASSURANCE

SURVEILANCE PLAN

(QASP)

Eastern Regional Research Center Operations and Maintenance Contract

#XXXXXXXXXXXXX

PREPARED BY:

Facilities Operations Specialist

USDA ARS ERRC

Wyndmoor, PA

November 11, 2023

Concurrence: ____________________ Date: ________ Contracting Officer Representative, ERRC

Reviewed by: _____________________ Date: ________ Contracting Officer, ARS/USDA

Version 1.1 Updated 11 NOV 2023

This page intentionally left blank.

Table of Contents

1. Introduction

1.1 Purpose

1.2 Partnering

2. Roles and Responsibilities

3. Training

4. Safety

5. Security

6. Submittals

6.1 Site Specific Quality Control Plan Submittal

7. Meetings

8. Methods of Assessment (MOA)

9. Quality Control Plan (QCP)

10. Performance Assessment Process

10.1 Post-Award Planning

10.2 Scheduling Assessments

10.3 IDIQ Task Orders

10.4 Assessment Procedures

10.4.1 AL1 Assessments

10.4.2 AL2/3 Assessments

10.4.3 Safety Assessment

10.4.4 Management and Administration Assessment

10.4.5 Contract Discrepancy Reports

11. Assessment Summary and Evaluation

11.1 Monthly Performance Assessment Summary (MPAS)

11.2 Invoice Validation and Withholdings

11.3 COR Activity File

11.4 Performance Assessment Board (PAB)

12. Summary

List of Attachments Attachment A: Functional Assessment Plan (FAP) Attachment B: Performance Assessment Worksheet Attachment C: QCP Pre-performance Review Checklist Attachment D: Customer Comment Record Attachment E: QCP In-process Review Checklist Attachment F: Contract Discrepancy Report (CDR) Attachment G: Safety Assessment Checklist Attachment H: Management and Administration Evaluation Checklist Attachment I: MPAS Coversheet Attachment J: PAB Rating Summary

Quality Assurance Surveillance Plan

1. Introduction

1.1 Purpose

This Quality Assurance Surveillance Plan (QASP) is established to monitor contractor compliance to complete the requirements listed in the performance-based Performance Work Statement (PWS) entitled “Operations and Maintenance Support Services for the Eastern Regional Research Center (ERRC) in Wyndmoor Pa.” This performance-based assessment plan sets forth the procedures and guidelines ERRC will use in evaluating the technical performance of the contractor. This requirement is for the full range of operational and maintenance support services at the ERRC campus. It is essential that the contractor providing this support be knowledgeable in a variety of maintenance and support functions and in providing a quality control plan keyed to the government’s performance requirements.

This plan shall be referred to as the Performance Assessment Plan (PAP). The PAP establishes Performance Assessment (PA) provisions for the Quality Assurance Surveillance Plan (QASP) of Facility Operation and Maintenance Support Services at the Eastern Regional Research Center (ERRC) Wyndmoor, Pa.

The PAP describes the methodology for assessing the Contractor’s performance that will be used to provide Contractor feedback and update the Contractor Performance Assessment Rating System (CPARS). The PAP includes the Functional Assessment Plan (FAP), Attachment A, standard Performance Assessment Worksheets (PAW), Attachment B, and Customer Comment Record (CCR), Attachment D, to document and report Government observations of Contractor performance.

The Government’s role is to assess Contractor’s work against measurable performance standards, and per the principles of Performance Based Services Acquisition (PBSA), the Contractor’s role is to ensure its quality through successful implementation of its QCP. Per FAR Subpart 46.4, Government PA “shall be performed at such times and places as may be necessary to determine that the supplies or services conform to contract requirements” in order to ensure payments are made only for services that meet performance standards specified in the contract.

1.2 Partnering

Effective partnering and establishing a positive relationship between the Government and the Contractor is essential in fulfilling a performance-based requirement. The Government’s relationship with the Contractor should be one that promotes a strong and positive business alliance to achieve mutually beneficial goals, such as timely delivery and acceptance of high-quality services through the use of efficient business practices. Business relationships should seek to create a cooperative environment to ensure effective communication between the parties. Teamwork, cooperation, and good-faith performance are important for meeting mission objectives and resolving conflicts and problems. Each party should clearly understand the goals, objectives, and needs of the other. It is essential that the Government and the Contractor work together as a team to communicate expectations, agree on common goals, develop a common understanding of measurable standards, and identify and address problems early in the contract to achieve desirable outcomes.

2. Roles and Responsibilities

The Government’s key roles and responsibilities for performance assessment are as follows:

Contracting Officer (CO). The Administrative CO and/or Procurement CO assigned to the contract. The CO has final responsibility for Contractor PA per FAR Part 42—Contract Administration and Audit Services, non-conformance modifications, and unilateral determination of incentives.

Contracting Officer’s Representative (COR). The COR is responsible for monitoring the Contractor’s technical compliance and progress based on the contract requirements specified in the PWS and in accordance with the PAP. The COR performs a variety of contract administration duties that includes oversight of PA, documenting and rating Contractor performance, reviewing invoices, and acceptance of work.

Facilities Operations Specialist (FOS). The FOS is the overall technical lead for the management of Facility Operations and Maintenance Support Services Contract requirements. The FOS may also function as the QAS.

Quality Assurance Specialist (QAS). The QAS is assigned as a Technical Point of Contact (TPOC) / Subject Matter Expert (SME) to the COR to perform duties as the on-site representative who assesses Contractor performance. The QAS periodically observes Contractor performance, reviews delivered services, reviews quality management corrective actions, periodically assesses and documents Contractor performance on Performance Assessment Worksheets (PAWs) and the Monthly Performance Assessment Summary (MPAS), and communicates findings as necessary with the Contractor, QAS, Safety/Security Officer, and the COR.

Note: Throughout ARS ERRC policy, processes, and training, the term Quality Assurance Specialist (QAS) refers to anyone responsible for conducting assessments of an ARS ERRC administered Facility Support Contract. The term QAS will also be used in reference to any individual assigned as a TPOC/SME to provide support to the COR, including as a collateral duty. All personnel assigned these duties must follow the guidance and direction provided to QASs.

Performance Assessment Board (PAB). The PAB is comprised of key technical and administrative personnel appointed in writing by the CO. The PAB will convene on a regular basis to review Contractor performance documentation for the prior evaluation period, and prepare and forward a summary report of findings and recommendations to the CO. The PAB makes recommendations for CPARS and provides input for the determination of contract incentives, if applicable. Details of PAB membership and the process for convening the PAB are provided in paragraph 11.4 below.

3. Training

To effectively implement the PA Program, individuals who monitor the Contractor’s performance should be experienced in the areas for which they are assigned and adequately trained.

CORs assigned to provide oversight of this contract must meet the applicable training requirements and must be appointed in writing by the CO per FAR 1.604. FOS/QASs providing support as TPOC/SME for the COR must meet the applicable training requirements and must also be assigned in writing.

Proper oversight of Contractor safety is an integral part of effective performance assessment. The FOS/QAS must ensure that the Contractor is in compliance with safety requirements specified in the contract. The FOS/QAS should be present during any local Safety briefings. If the FOS/QAS observes a violation of any safety requirements by the Contractor, the FOS/QAS should:

• Report the safety hazard resulting from unsafe acts or conditions, defective tools, materials, or equipment used by the Contractor to the COR and Safety and Security Officer.

• When imminent danger is apparent (where, if the hazard is not immediately corrected, there is a high probability that a serious accident will occur, life will be in danger or there will be extensive property damage), immediately inform the Contractor and request immediate action be taken to correct the hazard. If the Contractor does not voluntarily take corrective action, require the Contractor to stop work and immediately notify the COR.

Further detail of safety assessment procedures is provided in paragraph 10.4.3 below.

5. Security

The FOS/QAS should become familiar with all security requirements of the contract and report any observed violations to the CO and Security Officer.

6. Submittals

The FOS/QAS should review reports and other submittals identified in the contract to ensure they comply with applicable requirements and specifications IAW the PWS.

6.1 Site Specific Quality Control Plan (QCP) Submittal (spec 2.0.2) The Site Specific Quality Control Plan (QCP) Pre-Performance Review Checklist, Attachment C, should be used for the review of the Contractor’s QCP Plan submittal and as a guideline for discussion of the Contractor’s QCP during the post-award kickoff/pre-performance conference. The COR, QAS, Contractor Control Manager and Project Manager, and any applicable subcontractor quality representatives should sign off on the QCP review checklist.

7. Meetings

The FOS/QAS should attend and be prepared for required meetings, including partnering sessions.

The FOS/QAS should be familiar with the contract Spec Items covered in the meetings.

8. Methods of Assessment (MOA)

The FOS/QAS will periodically assess services for conformance to contract performance objectives and standards using the following MOAs:

• Periodic Sampling (PS) – requires a pre-determined plan for assessing a portion of the work, using sample size and frequency at the applicable assessment level.

• Validated Customer Comments (VCC) – consists of customers observing the performance of services they have received and using a pre-determined procedure to provide feedback and/or report observations to the FOS/QAS for validation.

• Unscheduled Visits (UV) – impromptu assessments of performance standards and objectives whenever practical.

• Customer’s Evaluation (CE) – consists of collected survey data of Contractor performance from the customer’s perspective through the use of a feedback form.

The MOAs used for assessment of each performance objective and standard are identified within the FAP included in Attachment A.

9. Quality Control Plan (QCP)

When the Government’s assessment of the Contractor’s performance reveals that the quality management efforts are not effective in ensuring performance objectives and standards are achieved, further action is required. The FOS/QAS will conduct a review of the Contractor’s QCP processes and quality inspection and surveillance records for the work item(s) where deficiencies are noted to validate the accuracy and effectiveness of the Contractor’s QCP.

For QCP to be considered acceptable, the Contractor must demonstrate to the Government through quality management and QC corrective and preventive actions that the risk of failure to meet performance standards has been satisfactorily mitigated.

Further detail of the QCP review process is provided within the assessment procedures in paragraph

10.4 below.

10. Performance Assessment Process

10.1 Post-Award Planning

Performance Assessment personnel should review and understand the final recurring and non-recurring contract requirements, including any amendments made during the solicitation period, paying particular attention to performance objectives and standards and any changes in the scope of work. Performance Assessment personnel should also review the Contractor’s technical proposal received in response to the solicitation and initial submittals, such as the QCP, Accident Prevention Plan (including Activity Hazard Analyses (AHAs) and Occupational Risk and Compliance Plans and Programs), list of key personnel and employee listing. (spec item #?)

Performance Assessment personnel should also meet with customer representatives to review details of the contract and discuss the process for reporting and handling of customer comments and review the contract requirements for partnering and the process to be prepared for these meetings.

10.2 Scheduling Assessments

Performance Assessment personnel should develop a planned assessment schedule based upon factors such as selected MOAs, Contractor’s recurring performance schedule, population of work, and local priorities and conditions. Certain work requirements may necessitate increased assessment based on performance risk considerations, e.g., services that are mission critical or have life safety impacts. Increased assessment may be conducted by adding Assessment Level 2/3 (AL 2/3) assessments or by targeting specific samples during routine Assessment Level 1 (AL1) assessment.

Risk is measured based on two things: the likelihood (or probability) an event will occur and the consequence (or impact) if the event does occur.

The FAP, Attachment A, along with the starting point for assessments based on risk determination should be compared against the Contractor’s work schedules as applicable to develop the initial assessment schedule. This schedule may be adjusted when required based on Contractor performance as detailed within the assessment procedures in paragraph 10.4 below.

10.3 Task Orders for Non-Recurring Work

Non-recurring work Task Orders (TO) require 100% assessment. This means that all non-recurring work (AW, CW) must be verified as satisfactorily complete prior to payment.

TO verification is performed by the customer. The customer must notify the COR by email or phone upon ordering a non-recurring work TO. The COR will schedule appropriate safety oversight for these evolutions. For all non-recurring work TOs, validation is the responsibility of Performance Assessment (PA) personnel. Scheduling of assessments must be planned based on the nature of the work (i.e. simple, short duration tasks performed at a single location vs. complex work performed over a longer period at multiple locations) and added to the assessment schedule after TO award.

10.4 Assessment Procedures

Every assessment must be documented on a Performance Assessment Worksheet (PAW) using the form provided in Attachment B. The assessment procedures based on the scheduled level of assessment performed are detailed below.

10.4.1 AL1 Assessments

The flowchart in Figure 1 below and corresponding descriptions shown below detail the initial performance assessment process used by the FOS/QAS to observe, assess, and document Contractor’s performance (AL1).

Figure 1. Performance Assessment Process for Assessment Level 1 (AL1)

Step 1: Assess Performance at AL1 – This is the typical starting point of assessment. Assess the Contractor’s performance using the MOA, frequencies, and sample sizes indicated at AL1 of the FAP.

The starting point may include additional PA at lower assessment levels for mission critical, safety, or environmental related services as determined based on the risk assessment performed during post-award planning. A Performance Assessment Worksheet (PAW) must be used for each assessment indicating this is an AL1 assessment. A PAW is the form used to document and report Government observations of Contractor performance.

Step 2: Standards Met? – Either the COR/FOS/QAS should evaluate the Contractor’s performance of work looking for both failures to comply with performance objectives and standards as well as instances of value-added services or work that exceeds performance standards. The Contractor’s performance will be deemed to have met standards if the work is substantially complete, meaning that any nonconformance is minor or trivial, there is no omission of essential work, and approximately 95% of the total work (population) assessed meets the performance standard. Substantial completion can be measured based on the total work requirement being assessed or based on any one element of work performance. However, any observation of work that fails to meet any of the specified performance standards will be documented on the PAW as a defect. DECISION: If performance standards are not met, continue. If performance meets standards, jump to Step 11. If performance exceeds standards, jump to Step 12.

Step 3: Document “Not Met” Assessment and Notify Contractor – Document results of assessment providing specific details of any observed negative performance that fails to meet contract performance standards, with supporting narrative on the PAW. If performance does not meet standards, the FOS/QAS will forward a copy of the PAW to the Contractor. The Contractor shall sign and return the PAW within the specified timeframe to acknowledge receipt of the document. The Contractor’s signature does not constitute agreement with the Government’s assessment, it merely acknowledges that the Contractor has been notified of a Government observed defect. Should the Contractor disagree with the Government’s observations, discussions should be conducted to reach a common understanding of performance objectives and standards.

Step 4: Rework if Necessary – In the case of unsatisfactory or non-performed work, the Government may, at its option, allow the Contractor an opportunity to correct by rework at no additional cost to the Government. Rework shall be completed within the timeframe specified in 48 CFR 52.249-8, Consequences of Contractor’s Failure to Perform Required Services clause of the contract. Any rework performed must be assessed and documented on the PAW.

Step 5: Defect(s) Warrant Evaluation of QCP? – Contractor performance warrants evaluation of QCP if: 1) defects are “Significant” or 2) a “Trend” of non-performance has been established.

Significant defects include the Contractor’s failure to meet performance objectives and standards that result in damage to the Government, or incomplete major or critical work items. Significant defects are subjective and should be discussed in initial partnering sessions with the Contractor. Trends of non-performance include defects that may not be significant but are recurring and have not been corrected through the Contractor’s QCP. Trends are typically defects found in the same or similar work requirements repeated consistently over several periods of the assessment frequency.

DECISION: If QCP evaluation is warranted, continue. If not, jump to Step 10.

Step 6: Evaluate QCP – The FOS/QAS should evaluate the Contractor’s QCP to verify proper controls are in place to ensure the delivery of quality services. The QAS should follow the QCP In- Process Review Checklist (Attachment E of the PAP) and document findings on this form. This review should begin with a focus on the Spec Items and/or location where defects have been found as opposed to a complete audit of the Contractor’s QCP (use Parts A & B of the checklist). The evaluation should identify corrective actions the Contractor is taking for specific discrepancies and identify any QCP changes the Contractor is implementing to preclude systemic problems, avoid repeat discrepancies, and regain Quality Control (QC). If the initial evaluation identifies deficiencies in the Contractor’s QCP with insufficient planned corrective actions or QCP changes, or, if corrective actions and QCP changes planned during previous QCP reviews have been ineffective, then broaden the evaluation to a more comprehensive review of the Contractor’s QCP program (use Parts C through F of the checklist).

Step 7: Recommend withholding if necessary – For any defect that leads to an evaluation of the Contractor’s QCP, withholdings may be warranted. The FOS/QAS should document recommendations for withholding of payment on the PAW for non-conforming services when these defects cannot be or have not been corrected by rework.

Step 8: Is QCP Acceptable? – The Contractor must demonstrate to the Government that they have taken corrective actions and identified QCP changes to preclude systemic problems, avoid repeat discrepancies, and regain QC. QCP is considered “Acceptable” if the Contractor’s actions will satisfactorily reduce the risk of continued failure to meet performance standards. DECISION: If QCP is unacceptable, continue. If QCP is acceptable, jump to Step 10.

Step 9: Add a lower Assessment Level for Spec Item and/or Location deficiencies – When the Contractor’s QCP has not adequately identified root causes for non-performance and implemented corrective action necessary to regain QC, additional PA at Assessment Level 2/3 (AL 2/3) should be conducted for the Spec Item and/or location deficiencies as shown in Figure 3. The FOS/QAS should discuss with the COR to determine if additional AL2/3 assessments are warranted. [End of this assessment] Step 10: Continue AL1 “Normal” Sampling – The FOS/QAS shall document all findings, including a summary of the findings associated with the Contractor’s QCP evaluation, on the PAW. The FOS/QAS should continue sampling the size identified as “Normal” in the FAP at AL1. [End of this assessment] Step 11: Document “MET” Assessment – Document results of assessment particularly noting what work requirements were assessed and details of the observations which indicate that performance complied with contract requirements providing any additional supporting narrative on the PAW. Jump to Step 13.

Step 12: Document “EXCEED” Assessment – Document results of assessment providing specific details of the work or performance that exceeded contract performance standards, with supporting narrative indicating the benefit to the Government on the PAW.

Step 13: Positive Trend Established? – If the Contractor has established a trend of performance that meets or exceeds standards, repeated consistently over several periods of the assessment frequency, the FOS/QAS should consider sampling at the reduced level (Jump to Step 15). If a trend has not yet been established the FOS/QAS should continue normal sampling.

Step 14: Continue “Normal” Sampling – The FOS/QAS should continue sampling the size identified as “Normal” in the FAP at AL1. [End of this assessment] Step 15: Consider “Reduced” Sampling – The FOS/QAS should adjust sampling to the size identified as “Reduced” in the FAP at AL1. [End of this assessment]

10.4.2 AL 2/3 Assessments

The flowchart in Figure 2 below and corresponding descriptions shown below detail the performance assessment process used by the FOS/QAS to observe, assess, and document Contractor’s performance for 3-digit and 4-digit Spec Items (AL 2/3).

Figure 2. Performance Assessment Process for Assessment Level 2 or 3 (AL2 or AL3)

Step 1: Assess Performance at AL 2/3 – Start additional assessment(s) at a lower level if QCP evaluation at AL1 was unacceptable. Certain work requirements may necessitate normal assessment at AL 2/3 based on performance risk considerations, e.g., services that are mission critical or have life safety impacts. Assess the Contractor’s performance using the MOA, frequencies, and sample sizes indicated at the appropriate assessment level, e.g., AL1 or AL 2/3 of the FAP.

Step 2: Standards Met? – The FOS/QAS should evaluate the Contractor to determine if all work is performed in accordance with the performance objectives and standards. The Contractor’s performance will be deemed to have met standards if the work is substantially complete, meaning that any nonconformance is minor or trivial, there is no omission of essential work, and approximately 95% of the total work (population) assessed meets the performance standard. If the assessed work is substantially complete, then a performance rating of Acceptable should be assigned. The FOS/QAS will document any instances of value-added services or work that exceeds performance standards with supporting narrative on the Performance Assessment Worksheet (PAW). When the assessed work fails to comply with performance objectives and standards, the FOS/QAS will document the defect on the PAW and notify the Contractor. DECISION: If performance standards are not met, continue. If performance meets standards, jump to Step 12. If performance exceeds standards, jump to Step 13.

Step 3: Document “Not Met” assessment and Notify Contractor – Document results of assessment providing specific details of any observed negative performance that fails to meet contract performance standards, with supporting narrative on the PAW. If performance does not meet standards, the FOS/QAS will forward a copy of the PAW to the Contractor. The Contractor shall sign and return the PAW within the specified timeframe to acknowledge receipt of the document. The Contractor’s signature does not constitute agreement with the Government’s assessment, it merely acknowledges that the Contractor has been notified of a Government observed defect. Should the Contractor disagree with the Government’s observations, discussions should be conducted to reach a common understanding of performance objectives and standards.

Step 4: Rework if Necessary – In the case of unsatisfactory or non-performed work, the Government may, at its option, allow the Contractor an opportunity to correct by re-performance at no additional cost to the Government. Rework shall be completed within the timeframe specified in Section E, Consequences of Contractor’s Failure to Perform Required Services clause of the contract. Any rework performed must be assessed and documented on the PAW.

Step 5: Defect(s) Warrant Evaluation of QCP? – Contractor performance warrants evaluation of QCP if 1) defects are “Significant”, or 2) a “Trend” of non-performance has been established.

Significant defects include the Contractor’s failure to meet performance objectives and standards that result in damage to the Government, or incomplete major or critical work items. Significant defects are subjective and should be discussed in initial partnering sessions with the Contractor. Trends of non-performance include defects that may not be significant but are recurring and have not been corrected through the Contractors QCP. DECISION: If QCP evaluation is warranted, continue. If not, jump to Step 11.

Step 6: Re-evaluate QCP – The FOS/QAS should reevaluate the Contractors QCP to verify proper controls are in place to ensure the delivery of quality services. This review should be limited to the Spec Items and/or location where defects have been found as opposed to a complete audit of the Contractor’s QCP. The evaluation should identify corrective actions the Contractor is taking for specific discrepancies and identify any QCP changes the Contractor is implementing to preclude systemic problems, avoid repeat discrepancies, and regain Quality Control (QC).

Step 7: Recommend withholding if necessary – For any defect that leads to an evaluation of the Contractor’s QCP, withholdings may be warranted. The FOS/QAS should document recommendations for withholding of payment on the PAW for non-conforming services when these defects cannot be or have not been corrected by rework.

Step 8: Is QCP Acceptable? – The Contractor must demonstrate to the Government that they have taken corrective actions and identified QCP changes to preclude systemic problems, avoid repeat discrepancies, and regain QC. QCP is considered “Acceptable” if the Contractor’s actions will satisfactorily reduce the risk of continued failure to meet performance standards. DECISION: If QCP is unacceptable, continue. If QCP is acceptable, jump to Step 11.

Step 9: Recommend appropriate administrative action – The FOS/QAS should make recommendations to the Contracting Officer via the COR for appropriate administrative actions.

Administrative actions may include additional performance review meetings, issuance of a CDR, withholding of payment including liquidated damages, or interim CPARS rating. The FOS/QAS should also document recommendations for withholding of payment on the PAW for non-conforming services when defects cannot be corrected by rework.

Step 10: Continue Current Assessment Level or go to lower Assessment Level if applicable – The FOS/QAS shall document all findings, including a summary of the findings associated with the Contractor’s QCP evaluation, on the PAW. The FOS/QAS shall continue sampling at the size and frequency identified in the FAP at the appropriate assessment level or can move to a lower level of assessment if applicable. Additionally, if there is a negative trend in Contractor performance, the FOS/QAS should consider modification of the MOAs, sample sizes, and frequencies included in the FAP. The FOS/QAS should discuss with the COR to determine if additional assessments or changes to the FAP are warranted. [End of this assessment] Step 11: Continue Current Assessment Level – The FOS/QAS shall document all findings, including a summary of the findings associated with the Contractor’s QCP evaluation, on the PAW.

The FOS/QAS shall continue sampling at the size and frequency identified in the FAP at the current assessment level. [End of this assessment] Step 12: Document “MET” Assessment – Document results of assessment particularly noting what work requirements were assessed and details of the observations which indicate that performance complied with contract requirements providing any additional supporting narrative on the PAW. Jump to Step 14.

Step 13: Document “EXCEED” Assessment – Document results of assessment providing specific details of work or performance that exceeds contract performance standards, with supporting narrative indicating the benefit to the Government on the PAW.

Step 14: Positive Trend Established? – If the Contractor has established a trend of acceptable performance over a period of time, e.g., three months, the QAS should discontinue lower level assessments (Jump to Step 16). If a positive trend has not yet been established the QAS should continue at the current assessment level.

Step 15: Continue AL 2/3 Assessments – The QAS should continue sampling at the size and frequency identified in the FAP at the appropriate assessment level. [End of this assessment] Step 16: Discontinue AL 2/3 Assessments – The QAS should discontinue the additional lower level assessment and move to a higher assessment level or reduce to normal AL1 assessment. [End of this assessment]

10.4.3 Safety Assessment

As detailed in ARS-ERRC policy, proper oversight of Contractor safety is an integral part of effective performance assessment. There are two preferred methods for assessing a Contractor’s safety performance: 1) Assessing safety while conducting regular periodic sampling; and 2) Documenting “unscheduled visits” to specifically assess safety anytime the performance of work can be observed.

Note: Anytime a safety issue is observed, the QAS should take appropriate immediate action to stop work as necessary until the unsafe practices are properly corrected.

The QAS shall use the Safety Assessment Checklist to record all safety assessments, identifying specific areas where safety issues were noted and including a supporting narrative regarding the safety issues observed in the comments block (attach checklist to the associated PAW). Similar to the assessment process detailed above, the QAS should consider the significance of safety issues and any trends observed in evaluating the need for further review of the Contractor’s safety program and the addition of more scheduled assessments.

If a detailed review of the Contractor’s safety program is deemed necessary, the QAS should verify proper safety controls are in place to ensure their employees are performing work in accordance with

OSHA regulations. This review shall ensure the Contractor’s safety program is site specific and relevant to the service process. The safety program review should identify discrepancies and identify any corrective actions the Contractor is implementing to preclude systemic problems and avoid repeat safety issues. The QAS should coordinate with the local Safety Representative for assistance in review of Contractor’s safety program.

The QAS must also be familiar with other safety responsibilities including assisting with Occupational Safety and Health Administration (OSHA) inspections and ensuring Contractors follow the proper procedure for mishap notification.

10.4.4 Management and Administration Assessment

Contractor compliance with contract requirements can generally be evaluated through the assessment of work performed. For example, the Contractor must provide properly trained and qualified personnel to perform work in order to meet the standards specified in the contract.

However, there remain certain overall management and administration requirements that cannot be effectively assessed through PA scheduled per the FAPs. Therefore, the COR will conduct a monthly assessment to evaluate the Contractor’s compliance with management and administration requirements using the checklist provided in Attachment H.

10.4.5 Contract Discrepancy Reports

Contract Discrepancy Reports (CDRs) are a formal administrative action intended to document and track Contractor corrective actions for resolution of continued unsatisfactory performance. CDRs will be issued for repeated failures where the Contractor has an unacceptable QCP that has not been effectively corrected. That is, the following conditions have occurred:

1) Defects at AL1 led to a QCP evaluation,

2) The Contractor’s QCP was found to be unacceptable and additional assessments were scheduled for the AL 2/3 level,

3) AL 2/3 assessments revealed further defects and the QCP evaluation was again unacceptable.

Issuance of a CDR requires the Contractor to evaluate the noted discrepancy, determine root cause of the failure to perform, and develop a plan to ensure contract requirements are met. CDRs require Contractor response and Government acceptance of the Contractor’s corrective action. CDRs must be tracked until officially closed out by the Government. The Contract Discrepancy Report format is included in Attachment F.

11. Assessment Summary and Evaluation

11.1 Monthly Performance Assessment Summary (MPAS)

The COR and FOS/QAS will collect, review, and evaluate the results of all performance assessments including PAW documentation, safety assessments, validated customer comments, customer evaluations, trend data, and Contractor QCP corrective and preventive actions. The QAS summarizes PA information and completes the comments block on the MPAS for each annex/sub-annex. The MPAS for each annex/sub-annex is included with the applicable FAP, Attachment A.

The COR reviews completed MPAS, provides recommended actions as applicable, assigns an overall technical rating for the function, and validates the MPAS by signing it. The COR consolidates all ratings with supporting comments on the MPAS Coversheet, Attachment I, provides a recommended overall rating for the Contractor’s performance, and validates the MPAS Coversheet by signing it. Supporting information (e.g. copies of completed PAWs, VCCs, Customer Evaluation forms, and other assessment documentation) should be made available with the MPAS.

11.2 Invoice Validation and Withholdings

Results of performance assessments and other PA information should also be used as part of the validation of the Contractor’s monthly invoice amount. The COR will make a determination for the value of the estimated damages to the Government for non-conforming or non-performed work and recommend to the CO the appropriate withholding including liquidated damages (LDs).

Documentation must be provided to support the reduced value of services and/or the estimated cost and related profit to correct deficiencies and complete unfinished work.

The COR is designated as a Departmental Accountable Official (DAO) due to the duties for invoice verification and the responsibility to ensure that payment recommendations are made only for services received that meet the performance standards of the contract. The COR must review the submitted invoices for accuracy and completion of required supporting documentation. The COR should reference MPASs with associated PAWs and other assessment documentation to verify completion of required services and determine if any withholdings or deductions are warranted.

For invoices submitted, the COR performs the inspector role and follows local process for documenting invoice reviews.

11.3 COR Activity File

In order to provide an auditable trail of documentation supporting the assessment of Contractor performance, the COR is required to maintain a file for each contract/order assigned. All contract documentation must be kept in the COR file. The COR File will be maintained until the end of contract performance, when it is then turned over to the Contracting Officer for inclusion as part of the official contract file.

Hardcopy/Digital files are maintained in a secure area by the COR in a folder(s) annotated with the contract number and period of performance for the included documentation. Supporting documentation (e.g. PAWs) for the current period of performance may be located in individual files retained by each QAS. All content in electronic format is located on a secure shared drive to be provided by the KO on a need to know basis.

11.4 Performance Assessment Board (PAB)

The Performance Assessment Board membership consists of the following:

PAB Chairperson – FOS PAB Member – QAS PAB Member – CO PAB Member – COR

The PAB will convene monthly to review and evaluate Contractor performance. The date, time, and location of PAB meetings will be established by the PAB Chairperson and communicated to all PAB members.

Additional participants may include Site Safety Manager and Customer representatives as specifically requested or approved by the PAB Chairperson. These personnel may participate in the discussion of Contractor performance but will have no vote on consensus ratings.

The COR (with support as required from FOS/QASs) should be prepared to brief the PAB on the monthly summary information and trend data and offer a recommended consensus rating to the PAB based on assessment results. Each PAB member should consider the information presented and individually document ratings with supporting comments for each area defined in CPARS on the PAB Rating Summary form, Attachment J. The PAB Chairperson should develop a consensus rating for each factor and document comments relevant to each rating factor from the PAB review.

At, or near, the end of each performance period, the PAB should review previous PAB Rating Summaries in addition to performance during the most recent evaluation period to develop overall input for official CPARS ratings and relevant comments. This final PAB report should be used by the Assessing Official Representative (AOR) for entry into CPARS for the performance period.

Additionally, this PAB should make final recommendations for assessing contract incentives in accordance with the Award Fee or Award Option Plan.

Specific details of the PAB process are provided in BMS B-14.26, Performance Assessment Board.

12. Summary

The PAP is based on the premise that the Contractor is responsible for managing and ensuring that quality controls meet the terms of the contract. The PAP facilitates consistent and effective tiered PA to verify the accuracy and completeness of the Contractor’s QCP and to assess overall compliance with performance objectives and standards. The Government will evaluate Contractor performance through appropriate assessment methods to ensure payments are made only for services that comply with contract requirements. This PAP is a “living” document that will be revised or modified as circumstances warrant.

FUNCTIONAL ASSESSMENT PLAN

CONTRACT NAME/#:

Attachment B

Attachment A: Functional Assessment Plan (FAP)

(see attached files)

PERFORMANCE ASSESSMENT

CONTRACT NAME/#:

Attachment B

Attachment B: Performance Assessment Worksheet

ASSESSMENT LEVEL � AL1 � AL2 � AL3 � TO

CONTRACT NO: QAS NAME:

SAMPLE ID: DATE:

SAMPLE LOCATION:

SPEC ITEM / TO #: TITLE:

Document Work Requirements Assessed:

SAFETY

SAFETY ASSESSMENT: Issues found? � No � Yes � N/A (attach Safety Assessment Checklist)

PERFORMANCE ASSESSMENT

� All work meets performance standards.

� Work does not meet standards (Provide specific details of the noted defects and recommendation of withholding if applicable.)

� Standards Exceeded (Provide specific details of the value added service including the benefit to the Government)

COMMENTS:

QAS (signature): DATE:

CONTRACTOR (signature): DATE: ______________

REWORK: � Acceptable � Unacceptable � N/A

QCP EVALUATION

COMMENTS: (Document effectiveness of contractor’s QCP to detect/correct negative performance and reverse trends. Attach QCP review checklist.)

QCP: � Acceptable � Unacceptable � N/A

RECOMMENDATIONS

COMMENTS:

QUALITY MANAGEMENT SYSTEM

PRE-PERFORMANCE REVIEW CHECKLIST

Attachment C

Attachment C: QCP Pre-performance Review Checklist

GENERAL INFORMATION

NAME PHONE EMAIL

CONTRACTOR Project Manager CONTRACTOR Quality Manager

SUB-CONTRACTOR QC

SUB-CONTRACTOR QC

FACILITY OPERATION

SPECIALIST (FOS)

QUALITY ASSESSMENT

SPECIALIST (QAS)

CONTRACT INFORMATION

TITLE:

Contract #: WO# LOCATION:

START: END: CONTRACT PRICE:

ACCEPTANCE OF CONTRACTOR’S QUALITY APPROACH DOES NOT LIMIT CONTRACTING

OFFICER FROM REQUIRING ADDITIONAL MEASURES IF PERFORMANCE IS

UNACCEPTABLE.

QUALITY MANAGEMENT BRIEFING CHECKLIST

CHECKPOINT (Y/N) COMMENTS

QUALITY ORGANIZATION:

Is the QCP submitted in accordance with the O&M Contract requirements?

Is the Quality organization clearly identified (e.g., org chart) and a list of all Quality personnel provided?

Are the responsibilities of Quality personnel detailed and lines of authority explained (e.g., Quality staff and Quality Manager reports directly to Prime Contractor management)?

Are the training and qualification requirements for Quality staff specified and does the Contractor’s staff meet these requirements?

Does the Quality organization show relationship between the Prime Contractor’s Quality staff and Subcontractor’s management or Quality?

Attachment C

QUALITY APPROACH:

Is the QCP current and specifically tailored for this contract?

Does the Contractor’s Quality Control Plan and management approach indicate a clear understanding of the contract requirements?

METHODS AND PROCEDURES FOR PERFORMANCE OF WORK:

Does the Contractor provide detail of their work planning and control to ensure first time quality?

This could include:

a. Proper selection and training of personnel

b. Tracking and verification of training and certification requirements

c. Work center supervisor/lead personnel oversight of work performance

d. Detailed SOPs and procedures for work requirements

e. Routine training and meetings

f. Selection procedures for subcontractors

g. Management control of subcontracted work

SURVEILANCE AND INSPECTION PROCEDURES:

Does the Contractor provide detailed procedure for the selection of samples (e.g., percentage of work inspected, process for selection of samples, in-process

vs. completed work.)?

Does the QCP detail procedures for the collection, recording, and analysis of inspection and surveillance results?

Does the QCP include processes for utilization analysis of inspection and surveillance results to determine cause and implement corrective actions?

Does the QCP provide a process for preventing recurrence of quality issues and continuous improvement of work performance?

Does the QCP detail specific procedures for the oversight of subcontracted work or the review and analysis of subcontractor quality?

Attachment C

DOCUMENTATION AND RECORDS MANAGEMENT:

Does the Contractor have a process for the control and retention of Quality documentation and records?

Does the Contractor provide the controls in place to ensure all Quality records are documented, maintained reviewed and properly filed?

Does the QCP have a process for the review of documentation for completeness, accuracy, and consistency? (This may include management reviews or internal audit plan.)

Does the QCP provide a process for tracking and ensuring all submittal requirements are met?

COMMUNICATION WITH GOVERNMENT:

Does the QCP address the level, format, and frequency of communications with the government? This could include:

a. Routine, yet informal communications between contractor, quality staff, and Government PA personnel.

b. Established meeting requirements between Contractor Quality and/or management staff with Government PA and/or contracting personnel.

c. Progressive reporting and communication based on the frequency or severity of the issue being addressed (e.g., Quality staff to QAS, Quality Manager to FOS, Project Manager to COR.

d. Details of protocol for attendance at meetings required by contract, including partnering sessions.

REVIEW SIGNATURES

FOS/QAS: DATE:

COR: DATE:

CONTRACTOR QUALITY MANAGER: DATE:

CONTRACTOR PROJECT MANAGER: DATE:

SUBCONTRACTOR: DATE:

SUBCONTRACTOR: DATE:

CUSTOMER COMMENT RECORD

Service Provided:

Attachment D

Attachment D: Customer Comment Record

CONTRACT:

DATE/TIME RECEIVED:

RECEIVED BY:

SOURCE OF COMMENT

ORGANIZATION: INDIVIDUAL: PHONE:

LOCATION:

WO #: TITLE:

DETAILS OF OBSERVATION: (Provide specific details of the requirement observed.)

Comment Validation: � Valid � Non-valid

COMMENTS:

FOS/QAS (signature): DATE:

CONTRACTOR (signature): DATE:

REWORK: � Acceptable � Unacceptable � N/A

FOS/QAS (signature): DATE:

QCP IN-PROCESS REVIEW CHECKLIST

Attachment E

Attachment E: QCP In-process Review Checklist

CONTRACT #: TITLE:

QAS NAME: DATE:

LOCATION:

SPEC ITEM: TITLE:

QCP REVIEW CHECKLIST

If observed defects warrant evaluation of QCP, the initial review should be limited to the contract specifications and/or location where defects have been found. This process begins with Part A & B.

CHECKPOINT (Y/N) COMMENTS

A. QUALITY SURVEILLANCE AND INSPECTION SCHEDULES

1. Is there a quality surveillance and inspection schedule?

Does it include:

a. Surveillance and inspections to be performed?

b. Frequency of surveillance and inspections?

2. Is there a current schedule?

3. Does the schedule reflect all contractual requirements?

4. Are the number and frequency of surveillance and inspections sufficient?

5. Do the schedules match the QCP?

6. Is the schedule being followed?

B. DOCUMENTATION AND ANALYSIS OF QUALITY DATA

1. Are the results of all surveillance and inspections properly documented?

2. Are quality deficiencies properly resolved and tracked?

3. Is quality documentation of deficiencies analyzed for trends and root cause?

4. Is appropriate action taken or planned to prevent recurrence of quality issues?

5. Is there verification process to ensure corrective and preventative actions are effective?

6. Are appropriate continuous improvement plans in place and communicated to workforce?

Comments: (Document corrective actions taken or QCP changes being implemented. If QCP is unsatisfactory, document findings and rationale for additional review conducted below.)

If review conducted above identifies deficiencies in the Contractor’s QCP with insufficient planned corrective actions or QCP changes, or, if corrective actions and QCP changes planned during previous QCP reviews have been ineffective, then continue review with Parts C through F below.

CHECKPOINT (Y/N) COMMENTS

C. QUALITY CONTROL PLAN

1. Is the written QCP available on site?

2. Is the QCP current?

3. Does the QCP staff meet the requirements designated in QCP plan (in terms of staff provided and qualifications and training)?

D. WORK PROCESSES AND PROCEDURES

1. Are work instructions, processes and procedures documented?

2. Are work instructions, processes and procedures available and used by affected personnel?

3. Is there a process to communicate work instructions, processes and procedures throughout the project and organization?

4. Are training records properly maintained for employees who are performing the work?

E. SURVEILLANCE AND INSPECTION PROCESS

1. Does the documented surveillance and inspection system match the requirements of the QCP?

2. Are surveillance and inspection forms used systematically that document both conformances and non-conformances?

3. Are the surveillance and inspection criteria linked to the performance objectives and standards of the contract?

4. Does the communication and follow-up on deficiencies follow the process detailed in the QCP?

5. Is analysis performed on surveillance and inspection data to identify trends and opportunities for improvement?

6. Are there examples of process improvements based on surveillance and inspection data?

CHECKPOINT (Y/N) COMMENTS

F. CUSTOMER COMMUNICATION

1. Are required meetings being held and attended as scheduled?

2. Is there documentation of the meetings and associated follow-up activities, i.e. action registers, meeting minutes, agendas?

3. Is there proper response and tracking of issues identified by Government personnel?

4. Is there a written documentation of issues, e.g., complaint/compliments logs, registers, records?

5. Is there a system for correction of defects/problems to satisfy customers?

6. Is there an escalation procedure if defects/problems are not addressed satisfactorily?

Comments: (Document corrective actions taken or QCP changes being implemented. If QCP is unsatisfactory, document recommendation to move to a lower assessment level or take appropriate administrative action.)

QCP RATING: � Acceptable � Unacceptable � N/A

REVIEW SIGNATURES

FOS/QAS: DATE:

CONTRACTOR QUALITY CONTROL REPRESENTATIVE: DATE:

Attachment F

Attachment F: Contract Discrepancy Report (CDR)

CONTRACT DISCREPANCY REPORT

1. CONTRACT NUMBER

GOVERNMENT ACTION

2. TO (Contractor…