J-2 Award Fee Evaluation Plan.pdf

Contact Center Training and Content (CTC) Fee Evaluation Plan

1 | P a g e

1.0 Introduction

The Contact Center Training and Content (CTC) contractor supports the efficient delivery of multichannel customer service for Medicare and Marketplace inquiries through the development and refinement of content and training, a focus on continuous improvement, and effective program management. This award-fee evaluation plan (hereafter referred to as the Plan) lays out the mechanism by which the CTC contractor will be evaluated.

2.0 Scope of Performance

The CTC contractor provides support across four workstreams:

• Content Development. Develop and update scripts and other relevant quick alert materials across all CSR communication channels in English and Spanish.

• Training. Develop and manage a training program, including computer-based, instructor-led, and train-the-trainer (T3) materials to increase the quality of customer service interactions in the contact center.

• Continuous Improvement. Continually assess and prioritize opportunities for improvements, including the effectiveness of content and training, and identify opportunities for increasing customer satisfaction, efficiency, or knowledge.

• Program Management. Maintain optimal staffing, coordination, and efficiency of the CTC program, including maintaining two-way communication with CMS and CMS-designated contractors to continually assess performance.

Additionally, the CTC contractor is responsible for supporting new initiatives, including inquiry types.

3.0 Evaluation Functional Areas

The following performance categories will be evaluated:

• Performance Metrics

• Content

• Training

• Continuous Improvement

• Program Management

• Contract Compliance

• Budget Planning, Adherence & Fiscal Responsibility

4.0 Award Fee Evaluation Plan

This Award Fee Evaluation Plan describes how the contractor will be evaluated and how the award fee process will be administered. The contractor shall satisfy the contract requirements by meeting or exceeding, when cost effective, all performance standards specified within the contract.

2 | P a g e

5.0 Performance Based, Cost-Plus-Award-Fee Description

The CTC contract is structured as a cost-plus-award-fee (CPAF) contract with the award fee elements determined by performance-based standards.

A cost-plus-award-fee contract is a cost-reimbursement type contract, which provides a tiered fee incentive to the contractor based on metrics and expectations. This fee consists of the minimum base fee fixed at award of the contract and an award fee that may be granted in whole, in part, or not at all, based on evaluated performance. The purpose of the award fee is to provide motivation for, and reward, excellence on the part of the contractor.

5.1 CPAF Contract Elements Descriptions

The basic elements of a cost-plus-award-fee pricing arrangement are as follows.

Estimated Cost The estimated cost is the contract cost negotiated and awarded and as subsequently adjusted via modification. The contractor will be evaluated on the level of variance between the negotiated estimated costs and the actual costs incurred by the contractor. The Government is committed to fiscal responsibility on behalf of the American people. The ability of the contractor to accurately estimate and reduce costs while maintaining service integrity and responsiveness is a direct reflection on the efficiency of services provided. CMS recognizes that unexpected outside influences—such as unexpected new policy initiatives—may affect the awarded estimated costs. The contractor is responsible for proactively notifying CMS when these situations occur, and the contractor will have the ability to explain cost variances +/- 5% at the CLIN level in monthly invoices and the bi-annual self-evaluations.

Base Fee The base fee represents the minimum profit for the contractor earned in performing the contract. The base fee will be defined at the time of award and is normally paid to the contractor on a scheduled, partial-payment basis throughout the life of the contract.

Award Fee The earned award-fee amount is determined based on the Government's evaluation of the contractor's performance. This determination is made unilaterally by the Government Fee Determination Official.

The maximum award fee pool that will be made available during the period of performance will be negotiated and established at contract award. The contractor is not guaranteed a minimum amount of award fee. In other words, some, none, or the entire maximum available award fee may be earned by the contractor. The Contract Specialist shall keep the contractor informed of the award fee pool available for each evaluation period.

3 | P a g e

6.0 Responsibilities

The Contractor and the Government have specific areas of responsibility in implementing, adhering to, and administering the Plan.

6.1 Contracting Officer’s Representative

The Contracting Officer’s Representative (COR) is the Government official designated to assess the Contractor’s performance as it relates to the performance standards. The responsibilities of the COR are to:

• Provide technical direction to the contractor and monitor the contractor's efforts

• Communicate on a regular basis with the contractor and discuss the contractor's performance

• Conduct progress review meetings with the contractor

• Evaluate the contractor's performance via a Performance Evaluation Report (PER) and present information bearing on the award fee recommendation to the Performance Evaluation Board (PEB)

6.2 Performance Evaluation Board (PEB)

The PEB is a panel of management officials who perform a review of the contractor’s performance and recommend an award fee to the Fee Determination Official (FDO). The PEB also recommends contract administration changes to the Contracting Officer and COR. PEB members will be appointed by the FDO. The responsibilities of the PEB are to:

• Approve the Plan

• Ensure that the award fee process is completed timely

• Perform a review of the contractor's performance three times a year

• Recommend the award fee to the FDO

• Substantiate the award fee in a report to the FDO

• Prepare an award fee letter

• Recommend contract administration changes to the Contracting Officer and COR.

The PEB will perform an in-depth review of the information provided by the COR, Contract Specialist, and Contractor along with any other pertinent information and shall arrive at the award fee recommendation. The PEB shall arrive at a consensus and shall prepare a written summary. This report is forwarded to the FDO.

The PEB may also review the Plan performance categories and weights and make recommendations and approval changes for future evaluation periods. Any changes shall be reviewed by the FDO and the Contracting Officer. The contractor shall be provided an updated version of the Plan prior to the effected evaluation period.

Any award fee recommendation may be more or less than the percentages recommended by the COR and Contract Specialist.

4 | P a g e

The use of weighted factors to calculate an award fee is an evaluation aid only and the total weighted scored does not necessarily represent the final earned award fee amount. The PEB has the discretion to recommend more or less than the total weighted score.

6.3 Fee Determination Official

The Government’s FDO shall make the final determination of the award fee. The FDO may determine a different award fee than what is recommended by the PEB; however, any change in the award fee shall be documented with explanation by the FDO.

This determination shall be made within forty-five (45) calendar days after the end of the evaluation period.

Upon receiving written notification of the award fee determination from the FDO, the contractor may submit an invoice for that amount.

NOTE: Unearned award fee shall not be carried over into subsequent evaluation periods.

6.4 Contracting Officer

The Contracting Officer is the Government official who bears the legal responsibility for the contract. The Contracting Officer alone can take action to enter into, terminate, or change a contractual commitment on behalf of the Government. The Contracting Officer will act in an advisory capacity during the award fee evaluation process and may also serve on the PEB.

6.5 Contract Specialist

The Contract Specialist is responsible for the day-to-day interaction with the COR and contractor in resolving contractual issues. The Specialist is also responsible for:

• Creating the Plan and amendments, and obtaining approvals

• Ensuring that the award fee process is completed timely

• Attending progress review meetings with the Contractor and the COR

• Evaluating the contractor’s performance in the following categories: Contract Compliance, Budget Planning, Adherence, and Fiscal Responsibility.

6.6 CTC Contractor

The CTC contractor shall:

• Successfully perform the specific responsibilities defined in the Statement of Work

• Provide complete and accurate information, monthly or as requested by CMS, for its performance to be effectively measured and evaluated

• Provide timely deliverables as specified in the Statement of Work

• Prepare a self-evaluation report and provide to CMS three (3) times a year

• Correct any deficiencies discovered by CMS during performance and documented in the Award Fee Evaluation Report.

• Make recommendations for award fee criteria updates for Government consideration.

5 | P a g e

7.0 Award Fee Criteria

The award fee will be determined according to performance among the following performance categories.

PERFORMANCE CATEGORY

Transition period of Base Year

WEIGHT

Base Year After Transition and Option Years

WEIGHT

TRANSITION IN (ADHERENCE TO PLAN AND MILESTONES) 80%

PERFORMANCE METRICS 20%

CONTENT QUALITY 5%

CONTENT AVAILABILITY 5%

TRAINING QUALITY 5%

TRAINING AVAILABILITY 5%

CONTENT 15%

TRAINING 15%

CONTINUOUS IMPROVEMENT 15%

PROGRAM MANAGEMENT 15%

CONTRACT COMPLIANCE 10% 10%

Budget Planning, Adherence & Fiscal Responsibility

10% 10%

TOTAL 100% 100%

7.1 Objective Performance Metrics

The objective metrics quantify and measure the contractor’s performance against defined performance metrics. The total value of the Objective Metrics Section is 20%. The Performance Metrics category consists of four primary metrics. (The performance metrics targets can be found in the Statement of Work under Section 4 “Performance Management.”

1. Content Quality (accuracy and completeness of content)

2. Content Availability (content developed within approved time frames)

3. Training Qualify (accuracy, training scores, train-the-trainer satisfaction, and CSR satisfaction scores)

4. Training Availability (training developed within approved time frames)

Non-compliance with any individual metric eliminates the entire award fee for that category for that month. Metrics shall be reported to CMS on a monthly basis. Compliance will be evaluated and the award fee determined on a monthly basis.

6 | P a g e

7.2 Subjective Criteria Description

In addition to the criteria listed in Section 7.1 above, the following criteria will be considered by CMS when scoring Content, Training, Continuous Improvement, Program Management, Contract Compliance, and Budget Planning, Adherence & Fiscal Responsibility. CMS reserves the right to modify criteria to reflect the current activities of the contract.

7.2.1 Content

The following criteria will be considered by CMS when scoring Content. CMS will consider such things as how well the contractor performed in resolving developing and revising content, identifying affected content, and responsiveness.

Criteria

Content

Reliable and consistent across all contact center channels and with other CMS communication (e.g., Healthcare.gov, Medicare.gov website)

Accurate according to CMS policy and procedures

Written to CMS plain language standards for ease of understanding by the CSR and the public.

Integrated into Content Viewer and presented in a format that supports maximum CSR performance effectiveness and efficiency (call flow)

Updated in a timely manner to ensure responsiveness to caller inquiries

Available in English and Spanish language versions

7.2.2 Training

The following criteria will be considered by CMS when scoring Training. CMS will consider such things as how well the contractor identified new training topics, developed training materials, and led train-the-trainer sessions.

Training

Reliable and consistent across all contact center channels and with other CMS communication (e.g., Healthcare.gov, Medicare.gov website)

Accurate according to CMS policy and procedures

Timely review and update of training materials with regulatory, policy, and other changes;

including rapid updates of any job aids or reference materials.

Employ best practices in training design and adult learning theory. Frequent use of “real-life” training activities

Incorporate interactive delivery methods and industry best practices

Meet minimum training satisfaction scores

7 | P a g e

7.2.3 Continuous Improvement

The following criteria will be considered by CMS when scoring Continuous Improvement.

Continuous Improvement

Identification – maintain cadence and inventory of identified opportunities

Planning and Prioritization - prioritize the opportunities identified and organized into a comprehensive plan

Execution

Monitoring and Measurement - Monitoring of implementation of changes and assess impacts against plan. Measure success/failure and any further recommendations in post-implementation analysis. Metrics based monitoring and measurement.

CSR Feedback - monitor and respond timely to CSR feedback related to training and content materials, as well as general CSR and customer sentiment about Medicare and Marketplace programs.

CSAT – Monthly satisfaction reporting and propose new survey questions when appropriate

7.2.4 Program Management

The following criteria will be considered by CMS when scoring Program Management.

Program Management

Management oversight of this contract including an efficient team structure, effective control systems, quality assurance, and reporting procedures.

Organizing and scheduling work to meet all agreed upon deliverables and engage in contingency planning to ensure continuous support.

Maintain strong working relationships with all CMS CCOG staff members and other stakeholders.

Maintain and ensure staff program knowledge, open communication with CMS to ensure understanding and approach prior to and during product development

Maintain coordination among related products across topics and programs. For example, ensuring changes to scripting are reconciled with the corresponding training products. Also, changes or enhancements to one program should be evaluated for appropriateness for the other program

7.2.5 Contract Compliance

The contractor will be evaluated on the ability to adhere to the requirements of the contract.

CMS will consider such things as how well the contractor met federal requirements & SOW-specific program requirements, SLAs, and performance targets and objectives.

8 | P a g e

Contract Compliance

The contractor will be evaluated on the ability to meet the contractual requirements of this contract, including but not limited to: compliance with federal statutes, contract clauses, Data Use Agreements (DUA), Joint Operating Agreements (JOA), HIPAA requirements, Privacy Act requirements, Security requirements, Business Continuity Requirements, and other requirements as designated by CMS or HHS.

Contractor’s business operations (e.g., accounting system, compliance with service contract, EEO, HIPAA, Privacy Act, Gov. Property System etc.) worked efficiently and effectively.

Contractor proactively worked with CMS to obtain approval prior to purchasing property and/or equipment, obtaining new subcontractors, and keeping the CMS contract specialist informed of programmatic issues which have contractual implications.

Contractor met all deliverable requirements with regards to submittal schedule, format and mode of delivery (e.g. electronic or hardcopy).

Contractor worked with CMS to establish preferred formats and layouts and to consolidate reports to make them more useful.

Contractor complied with all CMS security requirements, security standards and security regulations, policies and guidelines.

Contractor instituted best practices in implementing all security measures.

Contractor clearly identified all security deficiencies, developed corrective action plans, and validated that all corrective measures were implemented, tested and deemed effective.

Contractor complied with all CMS security certification and accreditation requirements.

Contractor complied with all security requirements with regards to building access and protection of property, sensitive information and employee wellbeing.

Contractor implemented best practices with regards to facility internal and external security with regards to safeguarding the integrity of PHI, PII information, and other sensitive information.

Contractor reported on and achieved performance metrics assuring accuracy and provided explanations for failure to meet any performance standards.

Contractor ensured, at a minimum, that program objectives were met by managing operations to meet the established performance metrics.

Contractor successfully met all performance indicator requirements and reported out providing explanations for variances between targets vs. actual with data and information to support the variances.

7.2.6 Budget Planning, Adherence & Fiscal Responsibility

The contractor will be evaluated on the Budget Planning, Adherence & Fiscal Responsibility including the cost effectiveness of the contractor’s performance. CMS will consider such things as how well the contractor estimated, tracked, analyzed & managed finances and planned for future variations and impacts to program operations

9 | P a g e

Budget Planning, Adherence & Fiscal Responsibility

Contractor developed a comprehensive budget plan which could be utilized to make clear decisions on how funds would be allocated.

Contractor ensured that budget planning process was based on a clear and concise strategy.

Contractor ensured that budget was an extension of resource planning (forecasting, staffing, scheduling, cost-analysis).

Contractor considered budgetary impacts in proposing recommendations for improving operational efficiencies and process improvements taking into account tangible and intangible benefits, time duration of initiatives and order of magnitude.

Contractor fully explained budget variances and provided mitigation strategies to prevent large impacts to various CLINS resulting from budget changes.

Contractor was proactive in identifying budget impacts, positive or negative, due to internal and external forces.

Contractor identified opportunities for cost efficiencies while still meeting program area goals and objectives.

Contractor cost estimates were accurate for the work performed.

Contractor proactively worked to stay within budget constraints and adhered to the established budget plan.

Contractor proactively identified and implemented cost savings while maintaining high quality service and meeting contract requirements.

Contractor proactively identified cost overruns and worked with CMS to mitigate cost increases.

Contractor was consistent in providing regular budget updates as well as backup information, as necessary, to make informed business decisions.

Whenever there is a significant cost overrun that was within the Contractor’s control, a contractor may be given a score of zero. If the overrun is insignificant, a higher score may be given. The reasons for the overrun and the Contractor’s efforts to control or mitigate the overrun will be considered in the evaluation.

Cost underruns within the Contractor’s control will normally be rewarded. However, the extent to which an underrun is rewarded will depend on the size of the underrun and the Contractor’s level of performance in the other award fee evaluation factors. The Contractor will not be rewarded for excelling in cost control to the detriment of other performance factors.

7.3 Award Fee Scoring Range

Performance will be evaluated according to the following standards. The weighted distribution will then be applied to determine the final award fee.

10 | P a g e

Grade Award Fee Pool Available

Description

Excellent 91% – 100% Contractor has exceeded all or almost all of the significant award fee criteria; and has met overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award-fee plan for the award-fee evaluation period.

Very Good 76% – 90% Contractor has exceeded many of the significant award fee criteria; and has met overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award- fee plan for the award-fee evaluation period.

Good 51% – 75% Contractor has exceeded some of the significant award fee criteria; and has met overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award- fee plan for the award-fee evaluation period.

Satisfactory No Greater Than 50%

Contractor has met overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award- fee plan for the award-fee evaluation period.

Unsatisfactory 0% Contractor has failed to meet overall cost, schedule, and technical performance requirements of the contract in the aggregate as defined and measured against the criteria in the award-fee plan for the award-fee evaluation period.

8.0 Contractor Self Evaluation

As part of the evaluation process, the contractor shall submit to CMS a self-evaluation report, not to exceed 15 pages in length, summarizing their activities in all evaluation categories during the period. The self-evaluation shall provide data for all performance metrics during the evaluation period, as well as narrative addressing any gaps in performance. Data shall not be aggregated or averaged for the evaluation time period but should be reported according to the reporting frequency required in the contract.

The Contractor shall submit reports no later than 10 calendar days after the end of each evaluation period.

11 | P a g e

9.0 Performance Evaluation Schedule

CMS shall make the award fee determination within 45 calendar days after the end of the evaluation period. The evaluation periods are as follows:

• Base Year: At the end of Transition (Will be based on the awardee’s proposed transition period) then twice more during the base year (e.g., Assuming a 6-month transition, the two evaluation periods would be 3 months each).

• Option Years - Every 4 months

10.0 Award Fee Evaluation Report

At the end of the performance period the COR and the Contract Specialist shall complete an Award Fee Evaluation Report. The COR may also choose to have other technical representatives evaluate the contractor’s performance.

The COR will rate the contractor's performance in each of the categories defined in this Plan.

The CORs assessment of performance will be designated by a numerical point rating system. A brief narrative supporting each rating shall be included.

The Contract Specialist is responsible for evaluating the contractor’s performance in the areas of Contract Compliance and Budget Planning, Adherence & Fiscal Responsibility with feedback & input from the COR. A brief narrative supporting the rating shall be included.

11.0 Converting Performance Points to Award Fee Score

The use of weighted factors to calculate an award fee amount is an evaluation aid. The total weighted score does not necessarily represent the earned award fee amount. The PEB has the latitude to recommend more or less than the total weighted score.

The PEB may choose, for example, to recommend earned award fee as follows:

1) The same percentage as the total weighted score

2) Somewhere in the range into which the weighted score falls (e.g., If the weighted score was 73 the panel could recommended anywhere in the “Good” range (i.e., 51 to 75%))

3) Something less than the total weighted score