Attachment_2_-_Quality_Assurance_Surveillance_Plan_Amendment_1.doc

RFQ CC15HQR0001

Attachment 2: Quality Assurance Surveillance Plan ATTACHMENT 2 - QUALITY ASSURANCE Surveillance PLAN Enterprise Managed Email Services (EMAIL) Solicitation

Quality Assurance Surveillance Plan (QASP) for the Enterprise Managed Email Services (EMAIL) Solicitation Note: The QASP is provided as a sample and is subject to change as a result of the Contractor’s Quality Control Plan (QCP). Upon award of the contract, the Contractor’s QCP will displace the QASP and become an integrated component of the PWS (the QASP is not part of the resultant contract).

June 17, 2015

Table of Contents

21.0

INTRODUCTION

21.1 Purpose

21.2 Performance Management Approach

21.3 Performance Management Strategy

32.0

ROLES AND RESPONSIBILITIES

32.1 Contracting Officer (CO)

32.2 Contracting Officer’s Representative (COR)

33.0

IDENTIFICATION OF SERVICES TO BE PERFORMED

34.0

METHODOLOGIES TO MONITOR PERFORMANCE

45.0

QUALITY ASSURANCE REPORTING

46.0

ANALYSIS OF QUALITY ASSURANCE MONITORING RESULTS

57.0

FAILURE TO PERFORM

6EXHIBIT I: PERFORMANCE REQUIREMENTS SUMMARY

8EXHIBIT II: SAMPLE OF QUALITY ASSURANCE MONITORING FORM

9EXHIBIT III: SAMPLE QUALITY ASSURANCE MONITORING FORM

1.0 INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) describes the procedures and guidelines the Office of the Comptroller of the Currency (OCC) will use to evaluate the technical performance of the Offeror in response to the OCC’s Performance Work Statement for Enterprise Email Services.

1.1 Purpose

The purpose of the QASP is to describe the systematic methods used to measure performance and to identify the artifacts and the resources that will support that performance measurement. The QASP provides a means for evaluating whether the Offeror is meeting the OCC’s performance standards.

This QASP defines the roles and responsibilities associated with performance measurement, identifies the performance objectives, defines the methodologies used to monitor and evaluate performance, describes quality assurance reporting, and explains how the results of quality-assurance monitoring will be analyzed.

1.2 Performance Management Approach

The OCC has provided a Performance Work Statement (PWS) that describes the work the Offeror will do in support of the OCC’s requirements. This QASP will define the performance management approach taken by the OCC to monitor, manage, and evaluate the Offeror’s performance against the expected performance objectives communicated in the PWS.

Performance management focuses on assessing whether or not outcomes are being achieved, and it de-emphasizes compliance with the processes and practices used to achieve the outcome. The only exceptions to process reviews are those required by law (Federal, state, and/or local) and compelling business situations such as safety and security. An outcome focus provides the Offeror flexibility to continuously improve and innovate over the course of the contract as long as the critical outcomes are being achieved at the desired levels of performance.

1.3 Performance Management Strategy

The Offeror’s Quality Control Plan (QCP) should specify the procedures for self-inspecting the scope of work subject to the performance requirements described in Exhibit 1 of this QASP. The Offeror shall implement and maintain a performance management system with processes to assess and report its performance to the Contracting Officer’s Representative (COR).

The COR will monitor performance and review performance reports furnished by the Offeror to determine how the Offeror is executing against communicated performance objectives. If necessary, the Offeror will be asked to modify its quality-control processes and practices to ensure that its performance is managed effectively.

2.0 ROLES AND RESPONSIBILITIES

2.1 Contracting Officer (CO)

The Contracting Officer (CO) is responsible for administering the contract, monitoring contract compliance, overseeing cost control, and adjudicating any disagreements between the COR and the Offeror regarding the Offeror’s performance. The CO will designate a full-time COR as the Government authority for performance management.

2.2 Contracting Officer’s Representative (COR)

The COR is responsible for monitoring, assessing, and communicating the technical performance of the Offeror to the CO and providing technical direction to the Offeror. The COR is responsible for completing the QA monitoring forms (i.e., Exhibits II and III) used to document the inspection and evaluation of the Offeror’s work performance. Government surveillance may occur under the Inspection of Services clause for any service relating to the contract.

3.0 IDENTIFICATION OF SERVICES TO BE PERFORMED

The Offeror shall provide all services in accordance with the OCC’s PWS. Exhibit I of this QASP identifies the performance requirements, performance standards, AQLs, and surveillance methods for the individual services that the OCC deems relevant for the purposes of quality assurance. The Offeror is free to augment these performance measures, if it wishes to do so; however, it should not modify any of the performance measures to render them easier to satisfy unless an explanation is provided for why the OCC’s performance requirement is not achievable at the specified level of quality. The Offeror is not permitted to propose additional incentives.

4.0 METHODOLOGIES TO MONITOR PERFORMANCE

Surveillance techniques shall be used by the Government to evaluate the Offeror’s performance. The primary methods of surveillance are random inspections, user feedback, and review of the records and files that are required to be maintained and delivered under the PWS. Feedback from the CO and COR, as well as input from OCC employees, will serve as the source of comments on the Offeror’s performance.

NOTE: OCC customers will be asked to communicate performance feedback to the COR as opposed to the Offeror. The COR will gather the customer feedback, validate the reported concerns, and record the feedback using the Quality Assurance Monitoring Form – Customer Feedback Investigation form provided in Exhibit III.

The performance standards described in Exhibit I, Service Performance Summary, are structured to allow the service provider to manage how the work is performed while offering incentives for performance that exceeds the required standards and imposing penalties for less-than-satisfactory performance. Levels of performance are aligned with the relative importance of the task to the overall effectiveness of support for the OCC’s EMAIL solution.

5.0 QUALITY ASSURANCE REPORTING

The performance management feedback loop begins with the communication of expected outcomes. The Offeror’s performance standards should be articulated in its PWS and aligned with the performance standards described in Exhibit I.

The Government’s QA monitoring, accomplished by the COR, will be reported using the sample monitoring forms in Exhibits II and III. Both the Offeror and the COR will maintain copies of the completed monitoring forms.

6.0 ANALYSIS OF QUALITY ASSURANCE MONITORING RESULTS

The Government shall use the observation methods cited to determine whether the performance standards have been met. The COR will evaluate the results and assess whether the performance standards have been met to determine if an incentive or penalty should be applied.

Upon request from the CO, the COR will prepare a written report to summarize the overall results of the quality assurance monitoring of the Offeror’s performance. This written report, which consists of the Offeror’s monthly performance self-report and the completed Quality Assurance Monitoring Forms (Exhibit II), will be maintained as part of the COR’s official contract documentation.

The COR may request that the Offeror’s Project Manager, or a designated alternate, meet with the CO and other Government personnel to discuss performance evaluations. The COR will define the frequency of these performance reviews with the Offeror; however, they will occur no more frequently than once a quarter unless an ongoing performance issue exists that requires more frequent consultation to facilitate resolution. The agenda of the review meetings may include discussion of the following topics:

· Monthly performance measured by the metrics and trends

· Issues and concerns of both parties

· Projected outlook for upcoming months and progress against expected trends

· Recommendations made by the COR and/or Offeror’s representative to maintain or improve performance

· Issues arising from the COR’s independent reviews and inspections.

In addition to conducting QA monitoring, the COR will use the information contained in the Offeror’s monthly self-report to assess the Offeror’s level of performance for each objective measured in this QASP (detailed in Exhibit I). The COR must coordinate and proactively communicate with the Offeror to resolve performance issues. The Offeror will discuss less than acceptable ratings with the COR and propose appropriate measures to improve performance.

The COR and Offeror should jointly formulate tactical and long-term courses of action. Deci-sions regarding changes to metrics, thresholds, or service levels should be clearly documented and mutually agreed. Such changes will be incorporated as a contract modification by the CO.

7.0 FAILURE TO PERFORM

The Offeror may be subject to penalties or contract termination based on a failure to perform. The following criteria shall apply for determining appropriate action:

1. Notifications. Consistent with FAR Part 49, the CO shall notify the Offeror of failure to meet standards. These notifications shall take the form of QA monitoring forms, cure notices, or show-cause notices.

2. Deductions. The Government has the right to assess penalties for the Offeror’s failure to meet performance standards. The specific penalties are identified in the Performance Requirements Summary (PRS) attached as Exhibit I.

3. Termination. If the CO determines that the Offeror has failed to perform to the extent that a termination for default is justified, the CO shall issue a notice of termination, consistent with FAR Part 49.

The Application of Incentives and Penalties

1. Both positive and negative performance will be documented. Penalties and incentives will be applied according to the standards defined in Exhibit 1.

a. In situations where performance does not meet the Acceptable Quality Standard (AQL) defined in Exhibit 1 but the OCC has chosen not to apply a financial penalty for unacceptable performance, the Offeror is expected to explain the failure to perform in its monthly self-report of performance. In addition to explaining the circumstances that resulted in the unacceptable performance, the Offer shall propose appropriate and timely measures to correct the deficiency and prevent a recurrence. The implementation of those measures will be subject to the COR’s approval.

2. Non-compliance with contract standards will be corrected by the Offeror at no cost to the government.

Notification. The Offeror shall respond, in writing and within three (3) business days of issuance, to all notifications by the CO of repeated failures to meet Acceptable Quality Levels (AQLs). The Offeror’s response shall clearly describe the corrective action(s) taken to rectify the problem and the proposed course of action to preclude the recurrence of the problem.

EXHIBIT I: PERFORMANCE REQUIREMENTS SUMMARY

Deliverables PRS:

Commence Full Scale Email Production Operation

Recurring PRS:

This performance standard is not applicable if there are no security events during the month.

(P) = 2% deduction in the monthly invoice when AQL is <95% for the month.

This performance standard is not applicable if there are no available patches or upgrades during the month.

(P) = 2% deduction in the monthly invoice when AQL is < 90% for the month.

This performance standard is not applicable if there are no available patches or upgrades during the month.

(P) = 2% deduction in the monthly invoice when AQL is < 90% for the month.

(P) = 2% deduction in the monthly invoice when AQL is < 85% for the month.

Ticket closure time for Tier II/III EMAIL -related tickets for VIP users

<= 30 minutes

(P) = 2% deduction in the monthly invoice when AQL is < 85% for the month.

(P) = 2% deduction in the monthly invoice when AQL is < 99.9% for the month.

EXHIBIT II: SAMPLE OF QUALITY ASSURANCE MONITORING FORM

SERVICE or STANDARD:

SURVEY PERIOD:

SURVEILLANCE METHOD (Check):

FORMCHECKBOX

Random Sampling

FORMCHECKBOX

100% Inspection

FORMCHECKBOX

Periodic Inspection

FORMCHECKBOX

Customer Feedback

LEVEL OF SURVEILLANCE SELECTED (Check):

FORMCHECKBOX

Monthly

FORMCHECKBOX

Quarterly

FORMCHECKBOX

As needed

PERCENTAGE OF ITEMS SAMPLED DURING SURVEY PERIOD:

ANALYSIS OF RESULTS:

OBSERVED SERVICE PROVIDER PERFORMANCE MEASUREMENT RATE =

SERVICE PROVIDER’S PERFORMANCE (Check):

FORMCHECKBOX

Meets Standards

FORMCHECKBOX

Does Not Meet Standards

NARRATIVE OF PERFORMANCE DURING SURVEY PERIOD:

Prepared By:

Date:

EXHIBIT III: SAMPLE QUALITY ASSURANCE MONITORING FORM

CUSTOMER FEEDBACK INVESTIGATION

SERVICE or STANDARD:

SURVEY PERIOD: ___________________

DATE/TIME FEEDBACK RECEIVED: _____________ ________ AM / PM

SOURCE OF FEEDBACK: _______________________________ (NAME)

_______________________________ (ORGANIZATION)

_______________________________ (PHONE NUMBER)

_______________________________ (EMAIL ADDRESS)

NATURE OF FEEDBACK:

RESULTS OF FEEDBACK INVESTIGATION:

DATE/TIME SERVICE PROVIDER INFORMED OF FEEDBACK: __________ AM / PM

ACTION TAKEN BY SERVICE PROVIDER:

RECEIVED AND VALIDATED BY:

PREPARED BY:

DATE:

� The 120-minute resolution period will not begin until the ticket is escalated by the OCC Tier-I staff to the Offeror’s Tier-II/Tier-III staff. Any delays or failures on the part of Government personnel (or OCC contractors who are not members of the Offeror’s team) to provide the support necessary for the Offeror to perform its duties in a timely fashion will invalidate this performance requirement.

� The 30-minute resolution period will not begin until the ticket is escalated by the OCC Tier-I staff to the Offeror’s Tier-II/Tier-III staff. Any delays or failures on the part of Government personnel (or OCC contractors who are not members of the Offeror’s team) to provide the support necessary for the Offeror to perform its duties in a timely fashion will invalidate this performance requirement.