Attachment 6 - Quality Assurance Surveillance Plan (QASP).pdf

ACF-GCS-S-0017-T07 (2020 version) Quality Assurance Surveillance Plan (QASP)

QUALITY ASSURANCE SURVEILLANCE PLAN (QASP)

State and Tribal Technical Assistance and Resources (STAR) Training and Technical Assistance for TANF Programs in support of:

Administration for Children and Families, Office of Family Assistance (OFA)

Solicitation: 75ACF124R00019

Quality Assurance Surveillance Plan (QASP)

ACF-GCS-S-0017-T07

STAR

1.0 INTRODUCTION

This Quality Assurance Surveillance Plan (QASP) is pursuant to the requirements listed in the Statement of Work (SOW) entitled State and Tribal Technical Assistance and Resources (STAR) Training and Technical Assistance for TANF Programs.

This QASP sets forth the procedures and guidelines the Administration for Children and Families Government Contract Services (ACF GCS) will use to ensure the required performance standards or services levels are achieved by the contractor.

1.1 PURPOSE

The purpose of the QASP is to describe the systematic methods used to monitor performance and to identify the required documentation and resources needed. The QASP provides a means for evaluating whether the contractor is meeting the performance standards or quality levels identified in the SOW and ensuring that the Government receives the services and products in the contract.

This QASP defines the roles and responsibilities, identifies the performance objectives, defines the methodologies used to monitor and evaluate the contractor’s performance, describes Quality Assurance (QA) documentation requirements, describes the analysis of QA monitoring results, and provides a Performance Requirements Summary (Attachment 1) that includes the method used to improve contractor performance when appropriate.

1.2 PERFORMANCE MANAGEMENT APPROACH

The SOW structures the acquisition around “what” service or quality level is required, as opposed to “how” the contractor should perform the work (i.e., results, not compliance). This QASP will define the performance management approach taken by ACF GCS to monitor and manage the contractor’s performance to ensure the expected outcomes or performance objectives communicated in the SOW are achieved. Performance management rests on developing a capability to review and analyze information generated through performance assessment. The ability to make decisions based on the analysis of performance data is the cornerstone of performance management; this analysis yields information that indicates whether expected outcomes for the project are being achieved by the contractor.

Performance management represents a significant shift from the more traditional QA concepts in several ways. Performance management focuses on assessing whether outcomes are being achieved and to what extent versus scrutiny of compliance with the processes used to achieve the outcome. A performance-based approach facilitates the contractor providing an innovative solution within the stated constraints that gives the Government the opportunity to receive outstanding results. A “results” focus provides the contractor flexibility to continuously improve and innovate over the course of the contract as long as the critical outcomes expected are being achieved and/or the desired performance levels are being met.

1.3 PEFORMANCE MANAGEMENT STRATEGY

The contractor is responsible for the quality of all work performed. The contractor shall develop and implement its own performance management system with processes to assess and report its performance to the designated Government representative. This QASP constitutes the Government’s approach to contractor performance.

Government representative(s) will monitor performance and review contractor products to determine how the contractor is performing against communicated performance objectives. The

Government will use this information when completing the Contractor Performance Assessment Report (CPAR). The contractor shall be responsible for making required changes in processes and practices to ensure performance is managed effectively.

2.0 ROLES AND RESPONSIBILITIES

2.1 ACF GCS CONTRACTING OFFICER (CO)

The ACF GCS CO is responsible for monitoring contract compliance, contract administration, and cost control and for resolving any differences between the observations documented by the ACF Program Office Contracting Officer’s Representative (COR) and the contractor. The ACF GCS CO will designate one full-time ACF Program Office COR as the Government authority for performance management. There will be approximately one additional representative serving as technical inspectors, due to the complexity of the services measured: Lisa Washington-Thomas, Division Director.

2.2 ACF PROGRAM OFFICE COR

The ACF Program Office COR is designated in writing by the ACF GCS CO to act as his or her authorized representative to assist in administering a contract. ACF Program Office COR limitations are contained in the written appointment letter. The ACF Program Office COR is responsible for technical administration of the project and ensures proper Government surveillance of the contractor’s performance. The ACF Program Office COR is not empowered to make any contractual commitments or to authorize any contractual changes on the Government’s behalf. Any changes that the contractor deems may affect contract price, terms, or conditions shall be referred to the ACF GCS CO for action. The ACF Program Office COR will have the responsibility for completing QA monitoring forms used to document the inspection and evaluation of the contractor’s work performance. Government surveillance may occur under the inspection of services paragraph contained in FAR clause 52.212-4(a) for any service relating to the contract.

3.0 IDENTIFICATION OF REQUIRED PERFORMANCE STANDARDS/QUALITY

LEVELS

The required performance standards and/or quality levels are included in the SOW and in Attachment 1, Performance Requirements Summary. If the contractor meets the required service or performance level, it will be paid for the time and materials. Failure to meet the required service or performance level will result in a negative past performance rating.

4.0 METHODOLOGIES TO MONITOR PERFORMANCE

4.1 SURVEILLANCE TECHNIQUES

In an effort to minimize the performance management burden, simplified surveillance methods will be used by the Government to evaluate contractor performance when appropriate. The primary methods of surveillance are:

a. Random Sampling: Performed by the ACF Program Office COR designated inspector.

b. Observation: Performed by the ACF Program Office COR on a weekly basis.

c. Periodic Inspection: The ACF Program Office COR typically performs the periodic inspection monthly.

4.2 CUSTOMER FEEDBACK

The contractor is expected to establish and maintain professional communication between its employees and customers. The primary objective of this communication is customer satisfaction.

Customer satisfaction is the most significant external indicator of the success and effectiveness of all services provided and can be measured through customer complaints.

Performance management drives the contractor to be customer-focused through initially and internally addressing customer complaints and investigating the issues and/or problems.

However, the customer always has the option to communicate complaints to the ACF Program Office COR, as opposed to the contractor.

Valid customer complaints must clearly set forth the detailed nature of the complaint, must be in writing and signed, and must be forwarded to the ACF Program Office COR. The ACF Program Office COR will receive those customer complaints and investigate them using the QA Monitoring Form: Customer Complaint Investigation, identified in Attachment 3.

Customer feedback may also be obtained either from the results of formal customer satisfaction surveys or from random customer complaints.

4.3 ACCEPTABLE QUALITY LEVELS (AQLs)

The AQLs included in the table in Attachment 1, Performance Requirements Summary, are structured to allow the contractor to manage how the work is performed while providing negative incentives for performance shortfalls. For certain critical activities, such as those involving the development and implementation of training and technical assistance (TTA) to Temporary Assistance for Needy Families (TANF) personnel and their partners the desired performance level is established at 100 percent. Other levels of performance are keyed to the relative importance of the task in relation to the overall mission performance at the Office of Family Assistance.

5.0 QA DOCUMENTATION

5.1 PERFORMANCE MANAGEMENT FEEDBACK LOOP

The performance management feedback loop begins with the communication of expected outcomes. Performance standards are expressed in the SOW and assessed using the performance monitoring techniques shown in Attachment 1.

5.2 MONITORING FORMS

The Government’s QA surveillance, accomplished by the ACF Program Office COR, will be reported using the monitoring forms in Attachments 2 and 3. The forms, when completed, will document the Government’s assessment of the contractor’s performance under the contract to ensure that the required results are being achieved.

The ACF Program Office COR will retain a copy of all completed QA monitoring forms.

6.0 ANALYSIS OF QUALITY ASSURANCE MONITORING RESULTS

6.1 DETERMINING PERFORMANCE

The Government will use the monitoring methods cited to determine whether the performance standards, service levels, and AQLs have been met. If the contractor has not met the minimum requirements, the contractor may be asked to develop a corrective action plan to show how and by what date it intends to bring performance up to the required levels. Failure to meet the AQL may result in a negative CPARS rating and exceeding the AQL may result in a positive CPARS rating, as shown in Attachment 1.

6.2 REPORTING

After each inspection, the ACF Program Office COR will prepare a written report for the PM summarizing the overall results of the QA surveillance of the contractor’s performance. This written report, which includes the contractor’s submitted monthly report and the completed QA Monitoring Form (Attachment 2), will become part of the QA documentation. It will enable the Government to demonstrate whether the contractor is meeting the stated objectives and/or performance standards, including technical/scheduling objectives.

6.3 REVIEWS AND RESOLUTION

The ACF Program Office COR may require the contractor’s project manager, or a designated alternate, to meet with the ACF GCS CO and other Government IPT personnel, as deemed necessary, to discuss performance evaluation. The ACF Program Office COR will define a frequency of in-depth reviews with the contractor including appropriate self-assessments by the contractor; however, if the need arises, the contractor will meet with the ACF Program Office COR as often as required or per the contractor’s request. The agenda of the reviews may include:

a. Monthly performance assessment data and trend analysis.

b. Issues and concerns of both parties.

c. Projected outlook for upcoming months and progress against expected trends, including a corrective action plan analysis.

d. Recommendations for improved efficiency and/or effectiveness.

e. Issues arising from the performance monitoring processes.

The ACF GCS CO and COR must coordinate and communicate with the contractor to resolve issues and concerns regarding marginal or unacceptable performance.

The ACF GCS CO, ACF Program Office COR, and contractor should jointly formulate tactical and long-term courses of action. Decisions regarding changes to metrics, thresholds, or service levels should be clearly documented. Changes to service levels, procedures, and metrics will be incorporated as a contract modification.

edited with fewer than five errors per five pages prior to submission and do not require government technical review with corrections exceeding 1 hour per five pages (formatting, spelling, grammar, etc.).

Contractor achieves 90% or higher rate of submission 15 calendar days of base year and each option year.

customer complaints

1.3 Revised Project

Management Plan

Contractor achieves a 90% or higher rate of submission of error-free plan that is proofed and edited with fewer than five errors per five pages prior to submission and do not require government technical review with corrections exceeding 1 hour per five pages (formatting, spelling, grammar, etc.).

Contractor achieves 90% or higher rate of submission within 10 calendar days of base year and each option year.

File reviews, periodic inspections, and random, observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

1.5 Project Meetings Contractor achieves

95% rate of success of participating in and planning in accordance with SOW requirements.

Inspections, and observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

1.5 Final Agenda and

Meeting Material

Contractor achieves a 90% or higher rate of submission of error-free agenda and materials that are proofed and edited with fewer than five errors per five pages prior to submission 24 hours prior to each meeting.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative Past Performance Rating in CPARS

1.5 Meeting Summary Contractor achieves

a 90% or higher rate of submission of error-free summaries that are proofed and edited with fewer than five errors per five pages prior to submission 48 hours after each meeting.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative Past Performance Rating in CPARS

1.6 Project Meetings

Agenda

Contractor achieves a 90% or higher rate of submission of error-free summaries that are proofed and edited with fewer than five errors per five pages prior to submission 24 hours prior to bi-weekly meeting.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative Past Performance Rating in CPARS

1.7 Priority Request Contractor achieves

a 90% or higher rate of response within 4 hours of priority requests during official business hours provided in

SOW.

Inspections, and observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

1.8 TTA Documents Contractor achieves

a 90% or higher rate of submission of error-free documents that are proofed and edited with fewer than five errors per five pages prior to submission 14 business days prior to a TTA event.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative Past Performance Rating in CPARS

1.9 Monthly reports

and vouchers

Contractor achieves a 95% or higher rate

Product reviews, periodic

Positive/Negative of submission of all invoices and monthly summaries with fewer than 2 errors no later than 7 calendar days before the end of the month.

inspections, and random, observations, customer complaints.

Rating in CPARS

1.10 Annual Reports Contractor achieves

a 95% or higher rate of submission with fewer than 2 errors per 5 pages no later than 7 business days before the end of contract year.

Product reviews, periodic inspections, and random, observations, customer complaints.

. Positive/Negative Past Performance Rating in CPARS

2.1 Develop a final

TTA plan

Contractor achieves a 90% or higher rate of submission of error-free plan that is proofed and edited with fewer than five errors per five pages prior to submission within 10 business days of meeting.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative Past Performance Rating in CPARS

2.2 Virtual Activities

and Resources

Contractor achieves a 90% or higher rate of submission of error-free materials that are proofed and edited with fewer than two errors per page prior to submission 15 business days after each event.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative Past Performance Rating in CPARS

2.3 Written Products, Tools, and Resources

Contractor achieves a 90% or higher rate of submission of error-free products that are proofed and edited with fewer than five errors per five pages prior to submission and do not require government technical review with corrections exceeding 1 hour per five pages (formatting, spelling, grammar, etc.).

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative

90% or higher rate of submission within 90 business days prior to due dates identified by the COR.

2.4 Information

Dissemination and Strategic Communications – Weekly email alerts Special Email Alerts

Contractor achieves a 100% or higher rate of submission of error-free weekly email alerts and other communications that are proofed and edited and resources verified and links checked for active status with no errors and do not require government technical review with corrections exceeding 1 hour per communication.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative Past Performance Rating in CPARS

3.0 TA Logistics and

Support

Registration and Meeting Materials

Contractor achieves a 90% or higher rate of completed hotel and meeting space negotiation/finalization six to nine months prior to, but no later than 90 days prior to the desired date of any in-person activity.

Contractor achieves a 75% or higher rate of completion of all air/ground travel planning no later than 60 days prior to the desired date of any in-person activity.

Contractor achieves an 75% or higher rate of completion of registration activities (development of save the date, creation of registration site/tracker) no later than 90 days prior to in-person activities.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative

Contractor achieves a 95% or higher rate of securing the most economical rates available for travel, lodging, and meeting costs that they are in compliance with the Federal Travel Regulations.

4.1 Facilities and

Equipment

Contractor achieves a 90% or higher rate of compliance with all federal requirements governing government furnished materials and access to federal facilities.

customer complaints

Positive/Negative Past Performance Rating in CPARS

4.3 Event Staffing Contractor achieves

95% of event staffing requirements as outlined in SOW.

Inspections, and observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

4.4 Post-event Activities Contractor achieves

95% completion of all post event activities in accordance with SOW requirements within 10 business days after each TTA event.

Inspections, and observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

4.4 Conference/meeting

spending reports

Contractor achieves 95% completion of all post event activities in accordance with SOW requirements within 10 business days after TTA event with no more than one error per page.

File reviews, periodic inspections, and random, observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

4.4 Post Event

Activities - Payments, honorarium

Contractor achieves 95% completion of all post event activities 30 business days after conclusion of meeting and 15 days thereafter for follow-up with fewer than 5 total errors.

Inspections, and observations, customer complaints

Positive/Negative

Rating in CPARS

4.5 Compile and analyze

Evaluation Results

Contractor achieves a 95% or higher rate of submission of completed evaluation summary report with fewer than 2 errors per

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative page 30 business days after the conclusion of the event.

4.5 Meeting Reports Contractor achieves a

95% or higher rate of submission of draft meeting reports with fewer than 2 errors per page 45 business days after the conclusion of the event.

Contractor achieves a 95% or higher rate of submission of final meeting reports with zero errors per page 14 business days after COR comments received.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative Past Performance Rating in CPARS

5.0 Expert Consultants Contractor achieves a

95% or higher rate of completed and signed consultant agreement and securement of consultants within 45 business days of COR request.

Inspections, and observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

6.1 Design, Maintain, and

Update the OFA PeerTA Website

Contractor achieves a 95% or higher rate of success of updating PeerTA website as outlined in SOW and in compliance with federal requirements within 30 business days post contract award and monthly thereafter with no more than one error per webpage.

Inspections, and observations, customer complaints

. Positive/Negative Past Performance Rating in CPARS

6.2.1 Develop and Maintain

Database

Contractor achieves a 95% or higher rate of success of developing and maintaining PeerTA website as outlined in SOW and in compliance with federal requirements 60 business days post contract award and within 5 days of the end of each month, thereafter with not more than one error per webpage.

Inspections, and observations, customer complaints

Positive/Negative

6.2.2 Website Metrics Contractor achieves a

95% or higher rate of development and submission of website metrics report as outlined in SOW and in compliance with federal requirements within 5 business days of the end of each month with no errors.

File reviews, periodic inspections, and random, observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

6.2.4 Website Performance

Plan

Contractor achieves a 95% or higher rate of submission of website performance plan 30 calendar days after contract award with no errors.

File reviews, periodic inspections, and random, observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

7.1.1 TTA information, website files, and code to transition the PeerTA website

Contractor achieves a 95% or higher rate of completion in accordance with SOW and federal requirements within 15 business days of contract award and with no errors.

Product reviews, periodic inspections, and random, observations, customer complaints.

Positive/Negative Past Performance Rating in CPARS

7.1.2 Contract Transition

Plan

Contractor achieves 100% rate of submission of a transition plan within 90 business days prior to the expiration of the contract award with no errors.

File reviews, periodic inspections, and random, observations, customer complaints

Positive/Negative Past Performance Rating in CPARS

7.2.1 Management-Bi-

weekly meetings with COR and other participants

Contractor achieves 95% or higher rate of participation in and planning related to task as outlined in SOW 30 business days after contract award and bi-weekly thereafter.

Contractor achieves 95% or higher rate of completion of all meeting planning activities (7.2.2) including agenda preparation and submission to the COR no later than two business days prior to each meeting.

customer complaints

. Positive/Negative

95% or higher completion and submission of meeting minutes within 24 hours after the meeting with not more than one error per submission.

ATTACHMENT 2

SAMPLE QUALITY ASSURANCE (QA) MONITORING FORM

SERVICE or STANDARD:

SURVEY PERIOD:

SURVEILLANCE METHOD (Check):

Random Sampling 100% Inspection Periodic Inspection Customer Complaint

LEVEL OF SURVEILLANCE (Check):

Monthly Quarterly As needed

ITEMS SAMPLED DURING SURVEY PERIOD: ___________________________

ANALYSIS OF RESULTS:

Service Provider’s Performance (Check): Meets Standards

Does Not Meet Standards

Narrative of Performance during Survey Period:

PREPARED BY: ___________________________________ DATE: _________________

ATTACHMENT 3

QUALITY ASSURANCE (QA) MONITORING FORM – CUSTOMER COMPLAINT

INVESTIGATION

SERVICE or STANDARD:

SURVEY PERIOD:___________________

DATE/TIME COMPLAINT RECEIVED: _____________ ________AM/PM

SOURCE OF COMPLAINT: _______________________________ (NAME)

_______________________________ (ORGANIZATION)

_______________________________ (PHONE NUMBER)

_______________________________ (EMAIL ADDRESS)

NATURE OF COMPLAINT:

RESULTS OF COMPLAINT INVESTIGATION:

DATE/TIME SERVICE PROVIDER INFORMED OF COMPLAINT: _________

_______AM/PM

CORRECTIVE ACTION TAKEN BY SERVICE PROVIDER:

RECEIVED AND VALIDATED BY:

PREPARED BY: ___________________________________ DATE: ____________