J-3 Statement of Work (SOW).pdf

Contact Center Training and Content

RFP: 75FCMC21R0041

Attachment J-3- Statement of Work

Procurement sensitive.

1 Introduction

2 Period and Place of Performance

3 Scope of Work

3.1 Content

3.1.1 Content Management Plan

3.1.2 New Content Development

3.1.3 Content Review and Updates

3.2 Training

3.2.1 Training Management Plan

3.2.2 Courseware Development

3.2.3 Courseware Review and Updates

3.2.4 Training Coordination and Delivery

3.2.5 Certification of CSR Performance

3.3 Continuous Improvement

3.3.1 Continuous Improvement Process

3.3.2 CSR Feedback Monitoring and Reporting

3.3.3 Customer Satisfaction Surveys

3.4 Program Management

3.4.1 Project Management Plan

3.4.2 Reporting

3.4.3 Project Kickoff

3.4.4 Staffing Approach

4 Performance Management

4.1.1 Performance Metrics -– Content

4.1.2 Performance Metrics - Training

4.1.3 Performance Metrics - Continuous Improvement and Program Management

5 Transition Approach

5.1 Transition In

5.2 Transition Out

6 Deliverables

7 Key Personnel

Procurement sensitive.

7.1 Senior Program Manager

7.2 Content Manager

7.3 Training Manager

7.4 Continuous Improvement Manager

8 Government-Furnished Equipment (GFE) & Information (GFI)

8.1 Content Viewer

8.2 National Data Warehouse (NDW)

8.3 Next Generation Desktop (NGD)

8.4 Learning Management System (LMS)

8.5 Verint Call Recorder

8.6 CMS Telecommunications Network

8.7 Joint Operating Agreement (JOA)

9 Security

9.1 Baseline Security Requirements

9.2 Training

9.3 Rules of Behavior

9.4 Incident Response

9.5 Position Sensitivity Designations

9.6 Homeland Security Presidential Directive (HSPD)-12

9.7 Contract Initiation and Expiration

9.8 Records Management and Retention

9.9 Privacy Act

9.10 Correct Deficiencies

9.11 Section 508 Compliance for Communications

10 CTC SOW Attachment

Procurement sensitive. 1

1 Introduction The Centers for Medicare & Medicaid Services (CMS) seeks a contractor to develop and maintain training and content for use by customer service representatives (CSRs) within its

Contact Centers of Operations (CCO). These training and content materials are used by CSRs to respond to inquiries. The training and content materials must be complete, accurate, and consistent across all contact center channels. Independently, and not as an agent of the

Government, the contractor shall furnish the necessary services, personnel, materials, equipment, and facilities, not otherwise provided by the Government, as needed to perform this contract.

CMS, an operational division within the U.S. Department of Health and Human Services (HHS), is the largest purchaser of health insurance in the United States. CMS helps to ensure high-quality healthcare at a reasonable price and to provide information about benefits, health promotion, and choices. CMS provides accurate, timely, relevant, understandable, and easily accessible information that helps individuals evaluate health plan options, nursing home choices, and make decisions on their individual health care needs.

On March 23, 2010, the President signed into law the Patient Protection and Affordable Care Act

(P.L. 111-148). On March 30, 2010, the Health Care and Education Reconciliation Act of 2010

(P.L. 111-152) was signed into law. The two laws are collectively referred to as the Affordable

Care Act (ACA). The Affordable Care Act creates competitive private health insurance markets

– called Marketplaces - that provide millions of Americans and small businesses access to affordable coverage.

To meet national program missions and strategies as well as legislative mandates (Balanced

Budget Act of 1997 and Medicare Modernization Act of 2003), CMS operates a toll-free, nationwide, 24x7 contact center with multiple sites to provide customer service and address inquiries using contractors. The inquiries that CCO responds to generally apply to the traditional

Medicare and ACA programs, as well as general questions about benefits, policies, referrals for additional assistance, and other individual healthcare needs.

CMS seeks a CCO environment that efficiently handles and responds to inquiries with a high level of service for residents across the United States and its territories. The contact centers span multiple contact channels, such as telephone, written correspondence, TDD/TTY, and web chat, across which consistently high service levels and quality must be provided. The nationwide toll-free lines (1-800-MEDICARE and 1-800-318-2596) provide beneficiaries and consumers with access to CSRs who are trained to answer questions regarding the Medicare and Marketplace programs. For Medicare, CCO responds to inquiries related to authorizations, benefit periods, claims (including denials, filing or status), election periods, deductibles, coverage, eligibility and enrollment, complaints, plan comparisons, prescription drug benefit enrollment and disenrollment, and appeal status (see below in section 1 for a full list of inquiry types). For the

ACA, CCO responds to inquiries regarding consumer health choices, authorizations, election periods, eligibility and enrollment, 1095 tax forms, complaints, plan comparisons, and appeal status. CMS is continuously exploring new options for streamlining processes and timeframes in the contact center to maximize efficiencies, cost-effectiveness, and caller satisfaction.

Procurement sensitive. 2

Annually, CMS experiences a “peak” inquiry volume from October through mid-January. This peak period is driven by high call volumes due to the Marketplace open enrollment period, mass mailing of the Medicare & You handbook, the Medicare annual enrollment period for prescription drug plans and Medicare Advantage Plans, and information sent by the Social

Security Administration (SSA) each December concerning cost of living adjustments and

Medicare premiums for the next calendar year. CMS may also experience unforeseen peak periods at other times in the year, based on changes in Medicare or other federal health program policy, legislation, media activities, etc.

CCO’s primary contact channels are telephone, web chat, and mailed letters in Spanish and

English. CCO utilizes language-line services for real-time translations in other languages.

Inbound interactions make up most telephone inquiries. Outbound telephone interactions are primarily handled by CSRs in Offline Research & Casework. Offline Research & Casework handles complex issues that require research and follow-up with the caller. Currently, approximately 1% of calls are escalated to Offline Research & Casework. Staff may also attempt outbound calls to resolve issues received via written correspondence or at other times during the year when the call volume exceeds the ability to respond to inbound inquiries.

Additionally, beneficiaries and consumers use self-service options such as interactive voice response (IVR) and the Medicare and Marketplace websites (https://medicare.gov, https://healthcare.gov), which drive unresolved questions to the contact center.

Contact center representatives use scripts to respond to most inquiry types. Inquiries denoted with an asterisk are both Medicare and Marketplace-related inquiries:

• Appeals Information*

• Authorization*

• Benefit Periods*

• Capped Rental

• Certificate of Medical

Necessity (CMN)

• Check Status

• Claim Denial

• Claim Filing

• Complaints*

• Coverage*

• Crossover

• Deductible

• Demographic

Information*

• Eligibility, enrollment, and conditions of coverage *

• Home Health

• Hospice Care

• Hospital Compare information

• Inpatient and

Outpatient Care

• Local providers, suppliers, and federal and state agencies*

• Medical Necessity

• Medicare Advantage disenrollment

• Medicare Advantage

Plan information

• Medicare Health Plan information

• Medicare Payment

• Mental Health

Services

• New programs or initiatives

• Nursing Home

Compare information

• Plan Compares*

• Prescription Drug

Benefit enrollment and disenrollment

• Prescription Drug information

• Preventive/Screening

Services

• Publication and print-on-demand requests*

• Referrals to outside agencies* https://medicare.gov/ https://healthcare.gov/

Procurement sensitive. 3

• End Stage Renal

Disease (ESRD)

Coverage

• Equipment Return

• Federal Marketplaces

• Fraud and Abuse*

• Medicare replacement card and address changes

• Medicare Secondary

Payer (MSP)

• Medicare Summary

Notice (MSN)

• Replacement

Equipment

• Small Business and

Employer inquiries

• Skilled Nursing

• Suppliers or

Providers*

The Contact Center Training and Content (CTC) contractor will be responsible for the following:

• Operating a responsive training and content development program that shares knowledge across the CCO team and rapidly incorporates new program initiatives and emerging/specialized needs into all operational procedures;

• Building the knowledge and skills of the CCO workforce through instructor-led, computer-based, and virtual training to improve the accuracy and quality of service for beneficiaries and consumers, their families, and caregivers;

• Developing, delivering, and maintaining scripts and other content that provide for high levels of customer satisfaction and transaction accuracy;

• Continuously measuring and strengthening the effectiveness of such materials using sources such as CSR feedback, customer satisfaction surveys, and call listening; and,

• Managing CTC activities, including interfacing with CMS and other CMS contractors, as required.

The CTC contractor shall maintain the currency of its knowledge of the Medicare program including Parts A, B, C and D; Durable Medical Equipment (DME), as well as trends and industry-leading practices for contact center operations. Additionally, the CTC contractor shall possess knowledge of the ACA and its Marketplace program.

The contractor shall work with CMS and other CMS contractors to assess the need for, develop, and administer training and content solutions. To do so, the CTC contractor shall implement an information-sharing framework across the team and functional areas. CMS views this knowledge sharing as key to successful operations.

2 Period and Place of Performance CMS anticipates award of a contract with a one-year base period plus four (4) one-year option years for a total potential period of performance of five (5) years.

The primary place of performance for this contract will be the contractor’s site, with travel required to CCO contact center sites for meetings and in person meetings at CMS Headquarters in Baltimore MD.

3 Scope of Work To support the efficient delivery of multichannel customer service for Medicare and Marketplace inquiries, the CTC contractor shall provide services across four simultaneous workstreams:

Procurement sensitive. 4

• Content Development. Develop and update scripts and other relevant quick alert materials across all CSR communication channels in English and Spanish.

• Training. Develop and manage a training program, including computer-based, instructor-led, and train-the-trainer materials to increase the quality of customer service interactions in the contact center.

• Continuous Improvement. Continually assess and prioritize opportunities for improvements, including the effectiveness of content and training.

Program Management. Maintain optimal staffing, coordination, and efficiency of the

CTC program in support of a 24x7 contact center operation, including maintaining two-way communication with CMS and CMS-designated contractors.

In addition to providing support for Medicare and Federally Facilitated Marketplace

(Marketplace) inquiries, the CTC contractor shall provide support for the rollout of new inquiry types related to new initiatives or program changes. The CTC contractor must understand CCO’s technology, regulatory, and knowledge management.

3.1 Content

The CTC contractor shall be responsible for developing, updating, and managing all content used by CSRs across all contact channels, in English and Spanish. All deliverables must be accurate, in plain language, and consistent across all channels. The contractor shall coordinate with other

CMS contractors in identifying content needs and to format and load content into the relevant system (see Section 8). All content must be approved by CMS prior to deployment to CSRs.

Typical items requiring development, updates and/or review may include:

Term Description

Scripts Collection of call scripts used by CSRs to resolve inquiries

Helpful Contacts

Database /Agent

Partner Search

External groups and associated programs that support

Medicare/Marketplace

User Alerts Messages developed to notify CSRs of script deployments or other changes.

Blast Messages Messages developed to notify CSRs of important or complex process changes. May include screen shots or images.

Code Definitions and Messages

Crosswalk of Next Generation Desktop (NGD) system codes to plain language.

Beneficiary and

Consumer

Messages

Messages tailored for specific beneficiaries and consumers based on various events related to them specifically (i.e., plan issue)

Written

Correspondence

Materials

(Paragraphs &

Letters)

Any written correspondence templates mailed or emailed to beneficiaries or consumers

CSR Feedback Use CSR feedback to improve process, collect and document issues uncovered internally or with Medicare/Marketplace

Procurement sensitive. 5

Term Description programs. Additionally, responding to feedback associated with training materials and scripted content.

IVR Language &

Queue Messages

Messaging created for the interactive voice response (IVR) system to relay information to callers

3.1.1 Content Management Plan

The CTC contractor shall develop a content management plan that governs the creation, review, revision, approval, and removal of content assets. Specifically, the CTC contractor’s plan will lay out the contractor’s approach to maintaining accurate, “plain-language” communication products across all channels. Specific plan requirements include the following:

• New content development o Categorization/classification of content, with corresponding file management approach o Develop process for new content, including rapid response requirements o Continuous delivery to CMS of impact analysis reports and action plans based on policy changes o Procedures for CMS review and approval of content o Approach to new content notification, user alerts, and blasts presented in Content Viewer

• Content review and updates o Schedule and procedures for review of existing content, including improvements to the scripting format, call flow, IVR scripting, etc., based on call reviews and research o Comprehensive change management approach that documents all content changes and provides an audit trail o Approach to validate and update on a quarterly basis or as directed/requested by CMS the helpful contacts referral database, which provides the referral information for the agent/partner referrals made by the CSRs and used by the CMS websites

• General o Spanish translations of all CSR scripts, IVR scripts, and other materials specified by

CMS

o Maintain compliance with Section 508 accessibility requirements o Use reasonable efforts to respond as quickly as practicable to unplanned critical requests to develop content and coordinate related activities and deployment with CMS and the

CCO during the CCO’s hours of operation, which are 24 hours per day, 7 days per week, and include holidays.

The contractor shall keep the plan up-to-date and provide such updates to CMS at least monthly or upon request.

3.1.2 New Content Development

In accordance with the content-management plan, the contractor shall develop CSR scripts and other materials to effectively respond to customer inquiries across all contact channels. This includes proposing new formats and approaches to improve content effectiveness.

Procurement sensitive. 6

For new content, the contractor shall conduct research to develop CSR scripts that provide a thorough response to beneficiaries’ and customers’ issues. The CTC contractor shall ensure that

CMS content is:

a) Reliable and consistent across all contact center channels and with other CMS communications (e.g., Healthcare.gov, Medicare.gov website);

b) Accurate in accordance with CMS policy and procedures;

c) Written to CMS plain language standards;

d) Integrated into Content Viewer and presented in a format that supports maximum CSR performance effectiveness and efficiency;

e) Updated in a timely manner to ensure responsiveness to caller inquiries;

f) Available in English and Spanish language versions; and,

g) Written to meet the Americans with Disabilities Act (ADA) Section 508 requirements.

3.1.3 Content Review and Updates

In accordance with the content-management plan, the contractor shall continuously conduct environmental scans of relevant statutes, policies, and regulations to identify outdated content for revision, alerting CMS of each required change. When a change has been identified or confirmed by CMS, the CTC contractor shall locate all affected materials, conduct research, perform impact analysis, develop the new scripts in Spanish and English, and secure CMS approval for the updates, then load into Content Viewer.

The contractor shall participate in post-implementation activities, including reviewing CSR feedback, analyzing script hit and other usage data from CMS’ National Data Warehouse (NDW) and Content Viewer web logs, and reviewing call audio and screen recordings as needed;

additionally, the contractor may be required to facilitate and participate in focus groups, and attend technical review sessions and other activities with CCO and CMS staff to better understand content requirements, assess current content, and develop new materials.

3.2 Training

The CTC contractor shall be responsible for developing, updating, and managing all training materials for instructor-led, train-the-trainer, and computer-based courses. Additionally, the CTC

Contractor shall coordinate with CMS and the CCO on training development and deployment throughout the calendar year.

All deliverables must be accurate and in plain language, and new training must be approved by

CMS prior to deployment to CSRs. Typical items requiring development, updates and/or review may include:

Term Description

Reference Material Documents to assist CSRs while handling inquiries (e.g., plan mailings)

Job Aids Quick reference guides for CSRs that outlines steps to resolution by category or contact type

Training Materials Instructor and student guides, course materials

Procurement sensitive. 7

3.2.1 Training Management Plan

The CTC contractor shall develop a training-management plan, addressing its training program for CCO staff, including CSRs and supervisors, and Offline Research & Casework staff.

Training shall also be available to relevant CMS staff and other stakeholders as directed by

CMS. The training plan shall address the following:

a) An evaluation of existing training materials and modality and any recommendations for evolving the materials and methods in which the materials are communicated;

b) Design and development of training materials and course-training roadmaps for a flexible training schedule and reduced in-class training time;

c) Review of Medicare and Marketplace program memoranda, CMS regulations, and other official documents for impacts to training materials or operational procedures and corresponding updates to training materials;

d) Development and maintenance of a system to track versions and document changes for all deliverables and provide a process for archiving materials;

e) A feedback and approval process for all new or updated training materials;

f) Weekly training plan updates with status of all materials in the development process and their dates of expected approval/delivery;

g) Timely delivery of assessment reports for train-the-trainer attendees and course-completion surveys from other students;

h) Participation in meetings related to development of new releases of NGD and CMS websites to determine any training needs; and,

i) Maintaining Section 508 accessibility requirements.

The training plan shall address regular updates to training materials and be flexible to evolving program, infrastructure, and technological change. Training comprises instructor-led courses delivered by the CCO contractor’s instructors in both in-person and virtual settings, train-the-trainer, and computer-based training. Training will cover the following three (3) broad topics:

• Desktop Application and Web-based Tools: The ability to use the computer-based tools

(Customer Relationship Management (CRM) and comparison tools available on various

Government websites, etc.) to effectively respond to beneficiaries’ and consumers’ questions; this includes communication tools such as IVR, web portals, and web chat.

• Customer Service: The interpersonal skills needed to communicate successfully with beneficiaries and consumers, including telephone etiquette and courtesy, deescalating angry callers, and effective written correspondence and web chat service skills.

• CMS Program and Policy: Knowledge of CMS programs and policies necessary to respond to Medicare and Marketplace inquiries.

Current training roadmaps for existing lines of business can be found in Attachment 1, Library of

Documents. At the direction of CMS, new lines of business may be established requiring updates to CTC training roadmaps. For example, the contractor may be required to update the training roadmaps to accommodate a change in how the CCO contractor manages its operations.

Procurement sensitive. 8

3.2.2 Courseware Development

The contractor shall develop training materials through various modalities including e-learning courses, instructor guides, participant guides, job aids, proficiency tests, and other reference materials for CSRs, supervisors, and other contact center staff. All approved training materials shall be posted to the CMS-provided Learning Management System (LMS). In support of the training topics in Attachment 1 and other CMS requests, the contractor shall:

a) Develop training materials, including those for Marketplace and Medicare new hires;

claims essentials; alternate channels; customer service skills; and process manuals for various tools, etc.

• Training materials for instructor-led training shall include both instructor guides and student materials.

• New curricula shall be pilot tested with a sample group prior to finalization and approval. CMS will work with the contractor to determine the pilot participants and sample size.

b) Regularly review and update training with regulatory, policy, and other changes; the contractor shall rapidly update any job aids or reference materials.

c) Employ best practices in training design and adult learning theory and incorporate CMS changes to materials. Frequent use of experiential, “real-life” training activities shall be included.

d) Curriculum needs to be adaptable to changes at the contact centers. For example, if CMS determines that calls should be handled by merging lines of business the current training materials will need to be adapted to account for the new process.

Incorporating interactive delivery methods, such as webinars, face-to-face and video teleconferencing sessions to facilitate updates to training materials is encouraged. For example, after receiving initial in-person training, participants may receive course updates through interactive delivery methods.

All training materials are subject to CMS approval prior to delivery of training. Most of the training materials require input and approval from CMS policy experts.

3.2.3 Courseware Review and Updates

The contractor shall review, update, and maintain training materials including instructor guides, participant guides, job aids, proficiency tests, and other reference materials for CSRs, supervisors, and other contact center staff. Courses may be in-person, remote for work-from-home CSRs, computer-based, or train-the-trainer. The contractor shall maintain version control of all courseware and ensure delivery of only the latest versions.

The contractor shall develop and maintain a process to track updates to training materials and provide a method for archiving materials. The contractor shall have access to the internal systems, including Next Generation Desktop (NGD) Production and Training environments.

The contractor shall identify areas of knowledge and skill deficiencies and develop new self-paced eLearning to refresh CSR knowledge, as necessary; in addition, the contractor shall

Procurement sensitive. 9 identify trends in knowledge retention and make modifications to base training materials, as necessary.

Additionally, the contractor shall survey a sample of CSRs to determine their satisfaction with the self-paced training eLearning. The contractor shall work with CMS and the CCO to determine the sample size and timing of pushing the survey through the LMS to the CSRs. The contractor shall monitor CSR satisfaction scores and comments for improvement opportunities.

3.2.4 Training Coordination and Delivery

The CTC contractor shall be responsible for coordinating with CMS and other contractors to design, create, and update materials, as necessary, ahead of scheduled training. In addition, the

CTC contractor shall provide train-the-trainer sessions for CCO contractor instructors on classroom courses developed. Train-the-trainer classes shall take place virtually, at CTC contractor site, or at a designated CCO contractor sites. The contractor shall be prepared to accommodate some ad hoc train-the-trainer sessions based on the addition of contact center locations and/or significant program updates.

While training is currently delivered primarily by the CCO contractor, the CTC contractor shall schedule and teach instructor-led training to the CCO trainers. The contractor shall be prepared to accommodate training delivery requests from CMS-approved users, such as CMS Regional staff members, Medicare administrative contractors (MAC), NGD users, and other CMS components. Classroom training may take place at the contractor’s site or designated CMS facilities and/or regional offices, and/or virtually, if needed. The contractor shall be responsible for providing the instructor, preparing the in-person or virtual “classroom”, including securing any audio-visual equipment or e-learning platforms required, and providing any handouts, student guides, or other materials.

All contractor-provided training shall include an end-of-class student-satisfaction survey. The contractor shall tabulate survey results and provide to CMS within 5 days of the conclusion of each class. Where necessary, the CTC contractor shall, in consultation with CMS, adjust course material. The contractor shall participate in post-implementation activities including gathering feedback from training, identifying lessons learned from the train-the-trainer sessions, and providing the feedback to CMS.

3.2.5 Certification of CSR Performance

The contractor shall use an assessment approach, developed in collaboration with CMS and other designated contractors that certifies individual CSR performance and meets CMS’s expectations.

Accordingly, the performance and learning progress of each CSR shall be assessed throughout the training sessions. At the end of the final training module, a comprehensive final exam shall be administered. CSRs must score at an acceptable level on the final comprehensive exam, 90% or better, before they may begin handling customer interactions in that channel. The contractor shall analyze assessment data for possible enhancements to the curricula.

Procurement sensitive. 10

3.3 Continuous Improvement

The CTC contractor shall identify continuous improvement (CI) opportunities for training and content, within the agreed parameters of a CI construct/cycle using a widely recognized methodology (e.g., Deming, Lean, etc.). The contractor shall consider all available data sources, including but not limited to CSR feedback, LMS, NDW, and IQA evaluations, to aid in the identification, prioritization, and measurement of implemented improvements. All deliverables must be accurate and in plain language, and all CI recommendations must be approved by CMS prior to implementation.

3.3.1 Continuous Improvement Process

The CI cycle shall be constant and should complete a full CI process within a time span decided upon by CMS and the contractor. Although the methodology will be at the contractor’s recommendation, the methodology/methodologies presented shall contain steps outlined below.

• Identification. The contractor shall establish a cadence and an inventory of means by which CI opportunities will be identified within cycle(s) of the chosen CI methodology.

The means of identification may include, but are not limited to, the following: focus groups, call listening, IQA evaluations, feedback from CSRs, feedback from operations, adoption of emerging trends, etc.

• Planning and Prioritization. The contractor will prioritize the opportunities identified;

after which, the opportunities are to be organized into a comprehensive plan. In planning for execution of the CI opportunities, the contractor shall treat each opportunity as a sub-project, which has all relevant Project Management Institute (PMI) management characteristics (e.g., integration, scope, cost, risk, etc.)

• Execution. The contactor will implement an approved list of improvements identified with CMS, including updating any relevant content, training, or standard operating procedures (SOPs). Should changes in the planned execution arise, the contractor is to stop work on the execution and review changes with CMS prior to proceeding.

• Monitoring and Measurement. The contractor shall provide a monthly recommendation tracking report to summarize and report status of all recommendations.

The contractor shall monitor implementation of the change(s) and assess impact(s) against plan. The contractor shall report on implementation success/failure and any further recommendations in post-implementation analysis reports Reporting on the impact(s) of implementation(s) shall be based on metrics.

3.3.2 CSR Feedback Monitoring and Reporting

The contractor shall monitor and respond timely to CSR feedback related to training and content materials, as well as general CSR and customer sentiment about Medicare and Marketplace programs. To help ensure feedback is reviewed and addressed by the appropriate contractor, the

CTC contractor shall develop a Feedback Management Plan that describes how the contractor shall:

• Use the CSR feedback captured in NGD to develop and facilitate a feedback process plan that incorporates CSR feedback along with customer feedback to health care

Procurement sensitive. 11 policy changes;

• Track, analyze, report and follow-up on the feedback provided by the CCO CSRs, supervisors, and other contractor personnel. Additionally, feedback responses shall be audited on a quarterly basis to ensure that the captured concerns are being properly addressed;

• Use feedback trends to alert CMS of possible changes needed to content and training.

All NGD feedback shall be trended to determine priorities and recommend measurable enhancements. These items shall be documented in the weekly and monthly reports;

and,

• Address and follow-up on training and content issues. The Contractor shall use the feedback tool to measure enhancements made to content/training and share successes documented in writing, as necessary to communicate such items to CMS.

3.3.3 Customer Satisfaction Surveys

The contractor shall monitor and analyze responses to the CMS-developed and Office of

Management and Budget (OMB)-approved Customer Satisfaction Survey. The survey is administered by the IVR and captures callers’ responses to questions at the end of their calls. The

IVR contractor then provides the survey results to the NDW contractor for reporting.

The contractor shall use the satisfaction survey results linked to data from individual calls to identify areas to increase customer satisfaction or to validate enhancements to scripting, training, or operations. The NDW provides reports and dashboards that include metrics that report customer satisfaction by contact center location, CSR tenure, average handle time (AHT), script hits, transfers, line of business, and identification and trending of low call scores and high call scores. The CTC contractor shall identify and share with CMS any areas of performance that merit recognition and consider root causes of dissatisfaction for potential improvements.

3.4 Program Management

The contractor shall provide management oversight of this contract including an efficient team structure, effective control systems, quality assurance, and reporting procedures. The contractor shall be responsible for organizing and scheduling work to meet all agreed upon deliverables and shall engage in contingency planning to ensure continuous support.

The contractor shall maintain a quality control plan that defines its approach, processes, and procedures for ensuring the quality and reliability of its products and services. The contractor shall maintain quality control of processes and procedures necessary to ensure its products and services meet the requirements of this contract.

The contractor shall maintain and ensure staff program knowledge along with open communication with CMS to ensure understanding and approach prior to and during product development.

Procurement sensitive. 12

The contractor shall maintain coordination among related products across topics and programs.

For example, the contractor shall verify changes to scripting are reconciled with the corresponding training products and changes or enhancements to one program are evaluated for relevance to the other program.

The contractor’s Senior Program Manager shall be the single contractor point of contact for all performance issues and be available for regularly scheduled and ad hoc meetings. The contractor’s Senior Program Manager or designee shall be available during business hours, 8:00 am to 6:00 pm ET, for coordination with the Government’s Contracting Officer, Contracting

Officers Representative (COR), other designated representative, and other CMS contractors. The

Senior Program Manager or qualified designee shall be available and accessible 24 hours a day, 7 days a week to address critical requests with CMS.

3.4.1 Project Management Plan

The contractor shall develop, maintain, and follow a project management plan (PMP) and supporting project schedule that includes its management strategy and processes for all content development, training, and related activities such as initiatives, pilots, and other endeavors as directed by CMS.

The PMP shall contain all deliverables, project tasks, staffing, the Contractor’s staffing approach and plan, and associated timeframes.

As part of the PMP, the contractor shall provide a work breakdown structure (WBS) and master project schedule. The project schedule shall highlight interdependencies and forecast critical dates. The contractor shall follow fundamental project management principles as highlighted in the Project Management Institute's Project Management Body of Knowledge (PMBOK). After approval, the contractor shall maintain the PMP for all areas of the contract on a weekly basis. At least weekly, the Senior Program Manager shall report to CMS any anticipated schedule delays and shall provide a recommended mitigation approach. The contractor shall report any schedule delays and recommended mitigations in the Weekly Status Report.

3.4.2 Reporting

The contractor shall be responsible for delivering periodic performance metrics and status reports to CMS. The contractor shall be responsible for the accuracy of each report and provide a written explanation to CMS for failure to meet any required performance standards.

A program progress report and a financial report shall be submitted monthly. The reports can be submitted as one report or two reports, so long as all the information is delivered to CMS. The financial report shall show the invoice breakout by contract line item number (CLIN) as well as by functional areas for the month. The report shall include a monthly staffing chart with personnel numbers associated with each functional area requirement to include total FTE count and total headcount. The report shall include the budget for each CLIN and functional areas as well as the dollar amount that was actually incurred. If the variance is over or under by 5% or more, the contractor shall provide an explanation as to what occurred to cause this variance.

Analysis shall be provided that includes the estimate to complete. The contractor shall also

Procurement sensitive. 13 provide cost trend information. For example, last year to current year or last month to current month.

The contractor shall provide status reports (as requested by CMS) at least monthly. This report shall identify key activities (completed or in progress) performed for all functional areas. It shall also describe any proactive steps that were taken to improve services, as well as identify any risks and mitigation steps that were implemented. The contractor shall address their performance against the performance metrics listed in the tables below (see Section 4). If a metric is not met, there shall be an explanation as to why it was not met. The contractor shall notify CMS of what happened last month, this month, and what is projected for next month.

3.4.3 Project Kickoff

The contractor shall conduct a project kickoff meeting within two (2) weeks of contract award to introduce key staff and their roles and responsibilities, assess and validate the current business requirements, define the project approach and scope, and provide a transition plan. The contractor shall be prepared to present to CMS its detailed approach and methods to develop, coordinate and manage project schedules, milestones, and deliverables for each of the project functional areas. The discussions held at the kickoff meeting shall include the following:

• Management Approach – assumptions and constraints for training, content, continuous improvement and the overall approach to managing the contract;

• Project Plan – the comprehensive methodology for achieving the detailed project schedule. The project plan shall include task descriptions, task dependencies, task durations, milestones, resources, deliverables, and identification of the critical path;

• Staffing Approach – the roles, responsibilities and allocations of each resource assigned to the effort; the approach for transitioning staff during various phases of the contract;

and the approach for estimating levels of resources required;

• Communication Approach – the methodology for identifying CMS stakeholders and needs, and communicating status of project goals, joint operating agreements (JOAs), issues, risks, and risk mitigation strategies to CMS stakeholders;

• Risk Mitigation Approach – the process, methods, tools, and resources that will be applied to the project for risk management. The contractor shall describe how risks shall be identified and analyzed, the basis for prioritizing risks, how risk responses shall be developed and implemented, and how the success of those responses shall be measured;

• Change Management Approach – the process for requesting, analyzing, prioritizing, and reviewing the impact of changes to established baselines, priorities, and documenting changes through their implementation;

• Quality Assurance Approach – the methods, standards, measurements, reviews, documentation of findings and schedule used to ensure the quality of the development process and products; and,

• Transition Approach – the plans for transitioning the business and technical processes for the training and content workload. At a minimum, the contractor shall provide roles, responsibilities, timelines, dependencies, risks, risk mitigation strategies, and milestones.

Procurement sensitive. 14

3.4.4 Staffing Approach

All CTC contractor personnel shall maintain program knowledge and shall be required to participate in the training that is developed and presented to the CCO CSRs. The CTC contractor shall present a training plan to CMS for its personnel to participate in training; the training plan may vary based on the designated role of the personnel; those participating in training developed for the CCO must pass the associated post-training testing upon the completion of training, and the contractor shall provide the test results to CMS monthly or as directed by CMS. Please see section 7 for key personnel.

The contractor shall develop and maintain a staffing plan to include the roles, responsibilities, and allocations of each staff resource assigned to the effort; the approach to transitioning staff between each life cycle phase; and the approach to estimating levels of resources required.

Staffing should be sensitive to, and specifically address, the range of requirements by program area. Staff should have the requisite knowledge, skills, and experience specific to the tasks to which the staff have been assigned. Any training necessary to educate the staff about the subject matter of the tasks should be conducted using the Contractor’s own resources and not at CMS’s expense.

4 Performance Management CTC contractor performance will be continuously evaluated by CMS to ensure that the overall project (content, training, and continuous improvement initiatives) meets performance management goals and objectives using monthly and weekly status reports. The contractor shall be responsible for the accuracy of each report and provide a written explanation to CMS for failure to meet any required performance standards. CMS will monitor the contractor’s performance to review the skills of all training and content staff and assess overall effectiveness.

Effectiveness will be evaluated using the following criteria:

• Knowledge of subject objectives and content

• Ability to convey subject matter in training material logically and clearly

• Ability to meet the objectives of the train-the-trainer session

• Communication skills

• Course satisfaction survey results and feedback

• Ability to present materials that actively involve trainees in learning experiences to reinforce lecture and course material

• Ability to meet timelines

4.1.1 Performance Metrics -– Content

Metric Definition Requirements

Reporting

Frequency

Accuracy and completeness of the content item

% of Content composed and submitted to CMS is both accurate and complete and meets CMS requirements

95% of content products are accurate, complete, and flow logically

Monthly

Procurement sensitive. 15

Metric Definition Requirements

Reporting

Frequency

Content Delivery % of Content should be developed and submitted for posting within CMS specified time periods

95% of materials are distributed to CMS and

CCO on-time

Monthly

4.1.2 Performance Metrics - Training

Metric Definition Requirements

Reporting

Frequency

Accuracy % of training product materials developed, updated and submitted to CMS is accurate, complete and flows logically

95% of training product materials are accurate, complete and flow logically

Monthly assessment of all products submitted

Training Scores Knowledge assessment for train-the trainer participants

90% of the participants receive a 90% or higher on the trainer assessment

Two weeks after train-the-trainer class

CTC Trainer

Satisfactions

Train-the trainer participant satisfaction

90% minimal training program satisfaction score

Two weeks after train-the-trainer class

Sample of CSR

Satisfaction

CSR participant satisfaction survey for self-paced training

90% minimal training program satisfaction score

Three weeks after eLearning was pushed to CSRs

CSR Satisfaction CSR participant satisfaction survey at the end of course-core curriculum training (new hire and claims)

90% minimal training program satisfaction score

Two weeks after end of course

Training Materials

Readiness

Training Materials are consistently and efficiently distributed throughout all training locations and national website

95% of training materials are distributed to CMS and the CCO on-time

Delivered to CMS in accordance with dates on project plan

Procurement sensitive. 16

4.1.3 Performance Metrics - Continuous Improvement and Program Management

Metric Definition Requirements

Reporting

Frequency

Effectiveness of

Ramp Up and

Delivery Time

Products to support new programs

Content and Training

Products produced to handle inquired in all communication channels

As needed

New Initiatives and

Approved Changes

Understand beneficiary/consumer requirements, manage scope and schedule

All changes managed through approved change processes

As needed

Contractor’s coordination and positive interaction across functional areas directly contribute to successful project operations

Contractor Key Personnel develop a partnership with

CMS and the CCO and embrace their goals and mission.

Training and Content products are delivered are in final form with no or few minor errors. Finished work is reviewed before submission for correctness and accuracy Key

Personnel are proactive in communication of project issues and responsive to stakeholder concerns

Quarterly

Monthly Program and Financial

Reporting

Dollars Spent vs. dollars budgeted

Project completed within approved cost parameters

Monthly

Weekly Project

Plan / Schedule

Updates

Tasks completed vs. tasks planned at a point in time.

Major milestones met

vs. planned. 90% of major milestones met.

Quarterly

Monthly Program and Financial

Reporting

Revisions to approved plan Understand and control project, all revisions reviewed and approved

Monthly

Customer

Satisfaction

Number of problems reported by CCO and CMS customers

100% of reported problems addressed within 72 hours, or in accordance with mutually agreed upon schedule

Quarterly

In addition to the performance metrics identified in the above tables, CMS requires that the contractor independently monitor and report on its performance. In the case that deviations in

Procurement sensitive. 17 performance result in lower quality and/or incomplete tasks, the contractor shall provide CMS with a report, stating the reasons for deviations from the listed objectives. In the future, CMS may identify additional metrics and objectives to monitor contractor performance.

5 Transition Approach The contractor shall support CMS and its designated contractors during transition into and out of the CTC environment. The contractor shall provide the following work products and deliverables in accordance with the functional requirements described in this SOW:

1) Transition Plan for a ramp up to include detailed milestone schedule, management plan, communication plan, training plan, contact lists and organizational charts for effective hand-offs of information and responsibilities between contractors.

2) System documentation materials for all government-furnished/contractor acquired property, property inventory, and lists of users and user account information.,.

5.1 Transition In

The contractor shall implement a phased-in approach to the transition. The contractor shall prepare all facilities, infrastructure, applications, and staff necessary to meet all requirements.

During the transition period, the contractor shall coordinate and support regular status meetings with CMS. During such status meetings the contractor shall, at a minimum, present to CMS accomplishments, issues, and potential risk areas. The contractor shall utilize a dedicated

Transition Manager, responsible for coordinating facilities and attendance. The Transition

Manager shall maintain continuity and quality of service and become familiar with CMS processes and methodology.

The contractor shall develop a Transition Plan that specifically addresses its methodology to prepare and execute its transition to complete the work outlined in this statement of work. This plan must be approved by CMS prior to deployment. The transition plan shall assume that the incumbent contractor will turn over government furnished information but will not turn over any government furnished equipment to support the contractor’s transition in period. The Transition

Plan shall address site-specific actions, methodologies, clearly defined processes, detailed schedules with realistic milestones, and specific actions to ensure continuity of service during transition. The Transition Plan shall also provide detailed risk analysis and mitigation strategies to minimize disruption of services. Additionally, the plan shall identify key events and critical milestone dates and shall detail any recommended parallel operations with existing services.

While the contractor’s Transition Plan shall address planning and events key to the contractor’s successful performance of transition, the contractor shall cooperate fully with CMS on the timely accomplishment of the transition plan. The COR or designated representative will coordinate transition efforts among current service providers and the contractor. CMS will also provide the contractor with the information, data, and applications necessary to transition.

Procurement sensitive. 18

5.2 Transition Out

At the end of the period of performance of this Statement of Work, the contractor shall work proactively with CMS and any other organization, as designated by CMS to ensure a smooth, orderly, cooperative transition of services. The contractor shall provide the most current versions of all materials, all data owned by the Government, and any final versions of materials created during this contract to CMS in a timely manner. During the transition of services, the contractor shall continue the work of this contract and operations at the same level of service.

The contractor shall submit a Phase-Out Plan that describes the contractor’s methodology, processes and phase-out transition activities that will be performed in cooperation with the incoming contractor. Work phase-out plans and delivery dates will be negotiated as soon as possible after notification of the new contractor’s proposed transition completion date.

6 Deliverables All deliverables shall be provided via electronic mail, or in hard copy as required. Deliverables shall be submitted in accordance with dates and time frames identified below.