Attachment_J.12-A_SOW_TO_0004.pdf

Attachment J.12-A SOW TO 0004

Measure & Instrument Development and Support

(MIDS)

Statement of Work

Task Order 0004

Measures Management System (MMS)

A. Statement of Work (SOW)

CHAPTER 1. SCOPE

1.A. - Background

The Centers for Medicare and Medicaid Services (CMS) is the federal agency tasked with overseeing a variety of health care programs, including Medicare and Medicaid. It strives to ensure that the American public receives the highest quality of care, consisting of personalized, prevention-oriented, and patient-centered care, based on evidence about the benefits and costs applicable to each individual patient.

The current health care system does not consistently deliver high-quality care for every patient at every opportunity, resulting in gaps in the quality of care provided for different population groups. Therefore, CMS recognizes the opportunity available to improve the health care system by closing such gaps, thereby further increasing the quality of care for our population.

One way that CMS will carry out its obligation to drive improvement in the health care system is through the development and use of quality measures and related activities. Quality measures are already widely used in several quality reporting programs, including the Hospital Inpatient Quality Reporting Program (IQR) and the Quality Payment Program (QPP). Quality measurement and quality reporting programs are also statutorily required or authorized for settings such as Hospices, Long-term Care Hospitals (LTCH), Inpatient Rehabilitation Facilities (IRF), Prospective Payment System (PPS)- exempt Cancer Hospitals, Psychiatric Hospitals, Hospital Outpatient Departments, and Ambulatory Surgical Centers (ASC) as well as value-based purchasing programs for hospitals, skilled nursing facilities (SNFs), and physicians, and End Stage Renal Disease (ESRD) centers. Other ways in which CMS carries out its obligation is through the standardization of core health assessment information in CMS’s post-acute care’s patient assessment instrument data, as required under the Improving Medicare Post-Acute Care Transformation Act (IMPACT) of 2014.

The Quality Measurement and Value-Based Incentives Group (QMVIG) in the Center for Clinical Standards and Quality (CCSQ) at CMS has developed a Measures Management System (MMS) for the development and maintenance of quality measures to be utilized to support the improvement of the quality of care for beneficiaries across all CMS programs.

The MMS continues to help CMS manage an increasing demand for high-caliber meaningful quality measures to use in its various public reporting and quality improvement programs to improve the effectiveness, efficiency, and services delivered to beneficiaries. In addition, stakeholders within the health care industry identified the need for accurate measures to evaluate the quality of care provided across the health care delivery system. Quality measurement is a critical tool for improving healthcare and for supplying information to consumers and purchasers in a market-driven health system. Performance measurement and quality improvement at the National level requires collaboration among federal agencies, health care facilities and multiple stakeholders to achieve significant change and improvement. In recent years, there has been a large and growing number of quality measurement and reporting initiatives resulting in an abundance of measures that are often duplicative and burdensome on the health care providers.

The lack of measurement standardization also leads to confusion for the public.

The Institute of Medicine (IOM) recommended that the Federal Government assume a strong leadership position in driving the health care sector to improve the safety and quality of health care services (Leadership by Example, 2002:

http://nationalacademies.org/hmd/reports/2002/leadership-by-example-coordinating-government-roles-in-improving-health-care-quality.aspx?_ga=2.138425587.1156372625.1516111600-1598524114.1516111600). Over the past several years, CMS responded to this recommendation by taking the lead in quality measurement and public reporting efforts. QMVIG, within CMS, is charged with this major responsibility and manages an array of quality programs that require the use of measures, including public reporting initiatives for hospitals, nursing homes, home health agencies, quality improvement initiatives and pay-for-reporting initiatives in the hospital and ambulatory setting, and value-based purchasing initiatives. QMVIG also serves as the CMS expert on quality measurement for demonstration projects, surveys and certification activities.

QMVIG also works closely with the “National Consensus Development and Strategic Planning for Healthcare Quality Measurement” contractor, the National Quality Forum (NQF) to obtain consensus endorsement of the measures used in CMS programs. The NQF has prepared a report (Addressing the Quality and Measurement Gaps, 2012:

https://www.qualityforum.org/Publications/2013/03/2012_NQF_Measure_Gap_Analysis.aspx ), which describes a multidimensional analysis of existing measures and identifies gaps in areas where useful measures do not exist. CMS will utilize the recommendations of this report in the Measures Priorities/Development Planning and Measure Tracking System, which will be applied to this proposed work

1.B - Purpose

The purpose of this MIDS TO SOW is to procure the expert services for the operation, maintenance and refinement of the MMS which aims to support the improvement of the quality of care for beneficiaries through the standardization of quality measure development, implementation, and maintenance.

This TO SOW adheres to all tasks within the MIDS Indefinite Delivery, Indefinite Qunatity (IDIQ) Statement of Work (SOW) Chapters. Those activities not specifically described in the MIDS IDIQ SOW shall be specified in this TO SOW. Those activities/tasks within the MIDS IDIQ SOW that are excluded or not required for this TO SOW shall be noted within this TO

SOW.

Given the purpose of this TO SOW, the MMS Contractor may not hold a CMS TO for quality measure development. However, the MMS Contractor may hold a non-measure development MIDS TO that allows them to maintain impartiality to the development process (ex. education and outreach or annual report). CMS will evaluate TOs on a case by case basis.

http://nationalacademies.org/hmd/reports/2002/leadership-by-example-coordinating-government-roles-in-improving-health-care-quality.aspx?_ga=2.138425587.1156372625.1516111600-1598524114.1516111600 http://nationalacademies.org/hmd/reports/2002/leadership-by-example-coordinating-government-roles-in-improving-health-care-quality.aspx?_ga=2.138425587.1156372625.1516111600-1598524114.1516111600 http://nationalacademies.org/hmd/reports/2002/leadership-by-example-coordinating-government-roles-in-improving-health-care-quality.aspx?_ga=2.138425587.1156372625.1516111600-1598524114.1516111600 https://www.qualityforum.org/Publications/2013/03/2012_NQF_Measure_Gap_Analysis.aspx

CHAPTER 2. GENERAL REQUIREMENTS

In accordance with (IAW) Chapter 2 of the MIDS IDIQ SOW, the contractor shall carry out all work/tasks described in sections 2.A. – 2.F, with the exception of 2.C.3 Section 2.C.3 (State Agencies for Survey and Certification) does not apply to this TO SOW.

Additional Requirements/More Specific Description for 2.C, 2.C.1, and 2.C.2.:

The Contractor shall attend a portion of the Kick-off meeting of new CMS TOs (up to 10 per year) to provide an introduction/overview of each section of the MMS and the use of the blueprint and its tools as requested by the CMS Contracting Officer’s Representative (COR). The Contractor shall provide a copy of the overview to the COR one week prior to these meetings.

The Contractor shall contact the appropriate COR to acquire the dates of all of these meetings and obtain a place on the agenda of the meeting. If there are more than 10 new CMS TOs awarded in a single year, the Contractor shall cover this information in a quarterly meeting with those awarded a new MIDS TO, in addition to up to 10 kick-off meetings.

Deliverable 2-6: Agenda and materials for supporting new CMS MIDS TOs

The Contractor shall make information related to measure development, testing, implementation, and similar processes available to other Measure Contractors (MC) upon request, as approved by CMS. The Contractor shall assume that technical assistance may be needed in the form of occasionally participating on conference calls with MCs, reviewing products from MCs and providing feedback to the MCs on these products. The Contractor shall be available to provide clarification to MCs regarding the measure evaluation process during the measure development of a new measure and reevaluation of implemented measures; steps to be taken in convening a Technical Expert Panel (TEP); steps to be taken in soliciting public comments on measures under development; steps to be taken to develop measures including developing complete technical specifications for the measures in accordance to the CMS Measure Information Form (MIF); provide clarification to the MCs regarding risk adjustment, measure testing, and measure implementation; provide clarification regarding the interactions necessary for the MCs to technically support measures through the NQF endorsement process. In addition, the Contractor shall support and maintain a system for collecting, responding to, and tracking questions from other MC.

The Contractor will act as a liaison and will provide communication, coordination, technical, and administrative support to provide CMS and MCs the ability to announce the following: calls for measures, nominations and updates for TEPs, and calls for public comment. The Contractor shall supply forms, collate received nominations, ensure forms and materials align with the Blueprint, and work with CMS staff to support the announcements on the MMS website, found at https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/MMS- Blueprint.html.

The Contractor shall conduct monthly MC measure development, maintenance and coordination meetings to ensure that the MCs are following the MMS Blueprint and to solicit information from CMS MCs regarding measure development activities for harmonization purposes.

The Contractor shall provide an agenda and meeting materials five (5) business days before the coordination meeting, and action items two (2) business days after the monthly meeting.

Deliverable 2-7: Summary of MIDS Contractor coordination and support Deliverable 2-8: Agenda, materials, and minutes of monthly MC call

The MIDS Deliverables Library is a repository of TO deliverables that MIDS contractors submit to the MMS Contractor as directed by their TO Schedule of Deliverables (SOD) or COR to promote sharing of best practices and lessons learned as well as avoiding duplication of effort and reduce cost. Due to the sensitivity of some of the deliverables, as not all deliverables are shared amongst MIDS contractors.

The Contractor shall use the files and metadata of the current MIDS Library to develop or procure a new MIDS Library system that maintains all security guidelines and protocols required by the Department of Health and Human Services (HHS) and CMS in order to maintain the system’s security compliance. This includes, but it not limited to maintaining an active Autority to Operate (ATO) and complying with the lastest CMS Acceptable Risk Safeguards (ARS).

The system shall include a web-based, single library structure containing all deliverables for all contractors while maximizing the flexibility for users to access deliverables using a variety of filters.

The Contractor shall maintain the MIDS Library to allow for user specific access so that their access only includes shared deliverables and their own deliverables. Users of the MIDS Library includes CORs/leads overseeing MIDS 2018 TOs and 2018 MIDS contractor personnel, as determined by the TO project director, even if they do not have an active 2018 MIDS TO. The Contractor shall receive and upload all new deliverable submissions. The Contractor shall maintain an inventory of all documents and facilitate periodic updates to the MIDS Library Sharing Plan. The Contractor shall provide technical assistance to all users as necessary to gain access to and use the system. The Contractor shall maintain a deliverable(s) request system, including staff of the MMS Contractor not specifically responsible for the MIDS Deliverables Library.

While this new system is being developed, the Contractor shall have an internal mechanism for supporting the functions of the MIDS Library, including receiving deliverables from MIDS contractors, inventorying deliverables, and responding to requests for delieverables.

Deliverable 2-9: MIDS Library System, CMS compliant Deliverable 2-10: MIDS Library Inventory Document Deliverable 2-11: MIDS Library Security Documentation

Additional Requirements for 2.D:

The Contractor shall provide a mid-year assessment of the project management plan using technicques like earned value management to. Updates to the budget and details of work to be performed. The format of the assessment will be determined in conjunction with the COR.

Deliverable 2-12: Mid-year Project Management Assessment

Additional requirements/More Specific Description for 2.F.:

The Contractor shall respond to approximately fifteen ad hoc requests per year from CMS.

These requests may be:

• Short-term and time-limited. Examples of these types of requests include:

o Analysis of opioid measures in the measures inventory o Alignment of measures in the inventory to the Meaningful Measures areas o Summary of meeting topics across the TO o Article for a CMS newsletter or blog about an MMS activity or measure development topic

• Long-term and more resource intensive. Examples of these types of requests include:

o review of measures in CMS programs to identify measures eligible for harmonization or termination from the program based on common clinical condition/topic, patient population, or other measure characteristics in order to simplify the public reporting burden of national providers.

o compile, review, and analyze: measure development and maintenance plans identified by existing and new CMS contractors and CMS program leads;

existing implemented measures for CMS quality reporting programs and measures identified by CMS for possible future implementation; planned measure output by NQF of specific measures and/or categories of measures for measure endorsement and maintenance o refining information and methods for measure developers: explore Failure Modes and Effects Analyses (FMEA) and how these might be integrated into the measure development process by outlining the steps required; develop detailed process map with stakeholder input

The Contractor response and required information shall be provided in a formal document that includes a one-page briefing summary suitable for presentation to CMS Leadership. Briefing summary delivered at the request of the COR.

CHAPTER 3. INFORMATION GATHERING

In accordance with (IAW) Chapter 3 of the MIDS IDIQ SOW, the contractor shall carry out all work/tasks described in sections 3.A. and 3.B. Sections 3.C— 3.E. do not apply to this TO

SOW.

Changes to the requirements of these sections 3.A. and 3.B. are specified below:

3.A. Environmental Scan:

The Contractor shall maintain and refine the Environmental Scan Support Tool (ESST) and Health Systems Harm Dashboard (HSHD) outlined below.

Environmental Scan Support Tool (ESST): The Contractor shall conduct a monthly environmental scan of major medical journals with reference to published articles and clinical practice guidelines that address CMS Quality Reporting Programs and the measures contained in these programs. CMS’ ESST increases the efficiency and effectiveness of the environmental scans required to develop and maintain measures by utilizing an automated Natural Language Processing (NLP) approach powered by CMS SematrixTM. Overall, the ESST is designed to rapidly scan an entire body of medical literature to:

• Identify relevant documents (abstracts and full-text articles), and

• Identify and extract the specific knowledge within each retrieved document that applies to the understanding of a potential opportunity for improvement that the measure is intended to capture.

Sematrix™ tags and analyzes published journal article using a training set of CMS Clinical Quality Measures (CQM) to find the most relevant articles related to each measure in the CMS measures inventory. Using an ontology to represent the knowledge contained in measure text, each article is given a relevancy score for each measure. The relevance score is a weighted average of the similarity in knowledge representation based upon keywords, biomedical concepts, biomedical concept expansions, semantic relations, and CQM model semantic relations. Citations from Measure Information Form (MIF) and the Measure Justification Form (MJF), gathered from the MIDS Library, for a subset of high impact and core measures are used to optimize the relevance score and provide a quality assurance assessment.

ESST is a part of the internal version of the CMS Measures Inventory Tool (CMIT). For more technical specifications, see 8.D. Program, Measure, and Initiative Assessment.

The Contractor shall maintain and refine ESST, including but not limited to:

• Providing monthly updates to the Content Management System of measure documents, abstracts and, preferably, open source articles

• Providing updates to CMS Sematrix™ Knowledgebase for all measures in the CMS Inventory

• Adding additional MIF citations to the Content Management System

• Automating the process of updating measure descriptions

• Conducting a scan quality assurance process

• Maintaining standard operating procedures (SOP), roadmap, and requirements documentation

• Implementing NLP re-training to improve quality and evaluating its impact

• Optimizing CMS Sematrix™ search weights and thresholds

• Soliciting stakeholder feedback from users and making improvements to the user experience as necessary

Deliverable 3-7: Monthly Summary Report of the ESST

Health Systems Harm Dashboard (HSHD): Literature evaluating unintended consequences of quality measurement was limited, of generally low quality, and inconclusive. As a result, CMS has identified challenges with intended consequence documentation, as an example, the MIF question 4.c: Unintended Systems Harms. HSHD is a tool for measure developers to use when responding to questions surrounding the unintended conquences of a quality measure. Measure developers can use the dashboard to help address questions about feasibility (pre-implementation) and monitoring (implementation) related to unintended harms by filtering Twitter data for content related to healthcare and collecting web content linked from Twitter into a single dashboard with tools for navigating the content. The tool facilitates information discovery by narrowing the scope of internet content to a subset of content containing potential harms and helping measure developers form hypotheses about potential harms that can be explored using peer-reviewed literature. HSHD uses National Library of Medicine’s MetaMap to identify change concepts, harm mentions, high impact conditions and high impact conditions.

MetaMapis designed to identify concepts from the United Medical Language System (UMLS) that occur in sentences. Each potential document identified from the Twitter data is mapped to Medical Subject Headings (MeSH®) terms using the MetaThesaurus. The document is then tagged with a topic if MetaMap identified a MeSHtopic associated with that topic.

The Contractor shall maintain and refine HSHD, including but not limited to:

• Monthly data collection and reporting

• Improving data migration performance

• Refining use and collection of Twitter data using the Test Twitter Free API

• Architecture consolidation and overall system performance improvement

• Tagging improvements and testing

• Maintaining a SOP, roadmap, and requirments documentation

• Soliciting stakeholder feedback from users and making improvements to the user experience as necessary

Below are design elements and requirements describing the system.

Design Elements of HSHD:

• Complies with the architecture of CMIT implementation and be available on the internal CMIT website with log-in

• Utilizes three-tiers (web server, static content, database) necessary to comply with CMS requirements

• Runs on the Java platform, utilizing J2EE technologies and patterns as needed

• Utilizes server-side Model-View-Controller (MVC) patterns as implemented via Struts-

Tiles

• Utilizes JQuery for DOM manipulation and custom scripting

• Utilizes Bootstrap for responsive user interface design

Data Collection Related Requirements:

• Tagging for harms will take a full text document as an input and return a list of text chunks with corresponding topic tags.

• URL collection will take a list of fully expanded URLs added to a cache from tweets, collect the data at the URL, and process it to prepare for tagging.

• Tweet collection will utilize the free streaming API available from Twitter using the HorizonScanning StreamHandler.

• Tweet collection will parse data from the stream table into the long-term data storage architecture.

Deliverable 3-8: Monthly Summary Report of the HSHD

3.B. Expert Input:

The Contractor shall provide technical and administrative support to various Workgroups and Task Force groups including, but not limited to the Measurement Policy Council (MPC) and the Quality Measurement Task Force (QMTF). These efforts are to assist CMS in compiling expert input from across CMS and HHS, as well as share CMS’ expert information, regarding measure planning activities, including standardizing measurement policies and framework, developing measures alignment strategies, developing core sets of measures across programs, and other related activities defined by the COR.

The Contractor shall support these Workgroups as follows:

• Compiling lists of work group members

• Scheduling meetings

• Documenting meeting attendance

• Providing meeting agendas and minutes

• Drafting documents for discussion as requested by CMS

• Providing lists of measures, measures cross walks and measure gap or harmonization analyses as requested

• Providing comments, analysis, and recommendations of materials as requested

• Provide the meeting materials 1 week in advance of the workgroup meetings for COR approval prior to workgroup distribution.

• Provide meeting minutes to the COR 5 business days after a meeting.

Deliverable 5-9: Workgroup meetings agenda, materials, and minutes

Additionally, the Contractor shall (in person or by conference call) participate in key measure developers’ meetings and conference calls where future measure needs and strategies are discussed, as directed by the COR. These meetings may be both internal and external to CMS.

The Contractor shall provide and submit a report of the proceedings 3 days after the meeting.

Deliverable 11: Expert Input Report

CHAPTER 4. QUALITY MEASURE DEVELOPMENT &

REEVALUATION

This Chapter and associated work and deliverables do not apply to this TO SOW.

CHAPTER 5. INSTRUMENT/ITEM DEVELOPMENT

CHAPTER 6. TESTING/VALIDATION

CHAPTER 7. REPORTS/APPROVAL PACKAGES

In accordance with (IAW) Chapter 7 of the MIDS IDIQ SOW, the contractor shall carry out all work/tasks described in sections 7.A. – 7.H.

Additional Requirements/More Specific Description:

7.E. — Periodic Status Calls

The Contractor shall arrange weekly conference calls with the COR and the program director, and other contractor staff, to review the overall TO and the details of the Project Management Plan and Work Plan, progress in tasks, barriers and mitigation strategies for completing the work, and other issues associated with the project work.

The Contractor shall also arrange weekly or biweekly conference calls with the CMS task lead(s) of the individual areas of work under this task to review the Tasks outlined in the SOW and the details of the Project Management Plan and Work Plan, progress in tasks, barriers and mitigation strategies for completing the work, and other issues associated with the project work. Examples of this would include a biweekly meeting between the MMS Blueprint CMS task lead and contractor task lead and staff or a weekly meeting between the measures inventory CMS task lead and contractor task lead and staff.

The Contractor shall provide a meeting agenda and related materials to the COR two business days prior to the call. Minutes or a summary of the call shall be submitted to the COR three business days after the call. The minutes shall include a list of all attendees, a high-level summary of the discussion, decisions, action items, and other pertinent information.

Deliverable 7-5: Periodic Status Calls Deliverable 7-6: Periodic Status Call Minutes Deliverable 7-8: Periodic Status Call Agenda & Meeting Materials

CHAPTER 8. IMPLEMENTATION, PRODUCTION & ASSESSMENT

In accordance with (IAW) Chapter 8 of the MIDS IDIQ SOW, the contractor shall carry out all work/tasks described in sections 8.A.1. (Measure Selection and Rule Making Process), 8.B.

(Measure Production), and 8.D. (Program, Measure, and Initiative Assessment). The remaining sections in Chapter 8 do not apply to this TO SOW.

Changes to the requirements of these sections are specified below:

8.A.1. Measure Selection and Rule Making Process:

The Contractor shall assist CMS with the Measures Under Consideration (MUC) by managing the pre-rule making process in its entirety; which is a statutorily mandated process by which CMS solicits and makes publicly available measures that will be considered for inclusion in multiple CMS programs. Annually, the final MUC list is published by December 1st and provided to the Measure Applications Partnership (MAP). The NQF-convened multistakeholder MAP is responsible for reviewing the measures and providing feedback on their appropriateness for use in the CMS programs. The Contractor shall provide expertise and technical assistance to CMS in areas that include but are not limited to:

• Preparation of documents and presentations for educational conference calls, webinars, in-person meetings for stakeholders, and similar activities;

• Participate in stakeholder education and outreach sessions, as well as the MAP meetings;

• Participate and provide technical support to CMS during the measure solicitation process that involves multiple stakeholders;

• Provide recommendations for continuous process improvements relative to the overall annual MUC process (e.g., measure tracking tool, education and outreach for stakeholders, and communications);

• Participate and provide technical support regarding CMS’ annual process of collecting candidate measures for consideration using the measure tracking tool (i.e., JIRA), which shall include but not limited to the input of all MUC, quality and completeness review of system measure submissions, system enhancements, system changes, system documentation, system training, and system reporting;

• On an ad-hoc basis, maintain and update the MUC list in the measure tracking tool providing drafts and final 508 compliant Measures under Consideration stakeholder reports for CMS;

• As requested by the MAP coordinators in collaboration with the CMS COR, provide measure reports as needed (e.g., July report that includes rule statuses);

• Review and provide quality assurance for the Measures Under Consideration List to ensure that all required information is included and clearly and completely described; and notify CMS of missing information or issues that need clarification;

• Maintain a list of all changes made to the MUC and the rationale;

• Draft and final documents, e.g., reports, charts, tracking logs, and tables related to the

MUC list;

• Participate in the review and editing process of the MUC list and any associated documents and reports;

• Participate on conference calls and meetings with the CMS COR as well as other CMS designated points of contact as needed;

• Participate in and assist with drafting appropriate responses during the different levels of the federal clearance process, including response to comments and questions from CMS entities and stakeholders; and

• Other duties as assigned Deliverable 8-7: Draft MUC List Deliverable 8-8: Final MUC List Deliverable 8-9: Agenda, materials, and minutes for MUC List related meetings

8.B. Measure Production:

The MMS Blueprint: The Contractor shall maintain, review, and update the various sections of the MMS Blueprint . The Blueprint has essential information that provides standardized guidance to Measure Contractors and CMS in the planning, development, implementation, reevaluation (maintenance), and harmonization of quality measures. The MMS Blueprint can be found at http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment- Instruments/MMS/MeasuresManagementSystemBlueprint.html.

The CMS MMS is a set of standardized processes and decision criteria that guide CMS and CMS measure developers in developing, implementing and maintaining the highest caliber of quality measures. This system is documented in a technical manual called the “Blueprint”. Each section of the MMS Blueprint shall be reviewed and updated at least twice a year or as directed by the COR, including any forms or templates to be used by CMS Measure Contractors (MCs) in developing/maintaining measures, in preparation for an update. The Contractor shall solicit comments from other MIDS contractors, and other pertinent stakeholders at the request of the COR (ex. NQF), at least once a year and incorporate those comments with guidance from CMS.

Deliverable 8-10 - A draft of the updated/revised MMS Blueprint and associated forms/templates completed twice a year, as directed by the COR.

Deliverable 8-11 - A final of the updated/revised MMS Blueprint and associated forms/templates completed twice a year, as directed by the COR.

The Contractor shall develop and maintain, in consultation with other CMS MIDS contract CORs, the MMS Blueprint COR Handbook, which complements the MMS Blueprint. It describes roles and responsibilities in managing the tasks of the MMS Blueprint and communication (issues, progress, risks, and Lessons Learned) between the MMS Contractor and CMS/QMVIG Leadership related to the Measure Development contracts. The Handbook ensures that CORs/ have a fundamental understanding of the measure development and maintenance processes.

Deliverable 8-12 - A draft of the updated/revised MMS Blueprint COR Handbook and associated forms/templates completed twice a year, as directed by the COR.

Deliverable 8-13 - A final of the updated/revised MMS Blueprint COR Handbook and associated forms/templates will be completed twice a year, as directed by the COR.

The MMS Blueprint and the NQF Consensus Endorsement Process: The Contractor shall review/update sections of the MMS Blueprint which address the NQF Consensus Endorsement http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/MeasuresManagementSystemBlueprint.html http://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/MeasuresManagementSystemBlueprint.html

Process to ensure that this process is aligned and current with NQF processes. The Contractor shall review the NQF processes and resources found on the NQF Web site (www.qualityforum.org) for the current NQF documents describing the consensus endorsement requirements and expectations and apply this information to update sections of the MMS Blueprint.

The Contractor shall identify where and how the work of NQF can be coordinated with CMS measurement development and implementation needs.

The Contractor shall participate with the NQF and major measure developers, as requested by CMS, to develop consensus regarding Measurement Policies and Methods and work to harmonize initiatives.

The Contractor shall participate in meetings between CMS and NQF in order to ensure coordination on endorsement, maintenance, and updates to the MMS and on NQF processes up to 6 times a year. The Contractor shall provide suggested agenda items for the meetings to the COR at least 2 week before the meeting and minutes 7 business days after the meeting.

Deliverable 8-14: Agenda, materials, and minutes for NQF meetings

General MMS Support and Education for CMS and Measure Developers:

The contractor shall support the continued learning of quality measures and measure development topics to CMS stakeholders (MIDS measure developers, non-CMS contracted measure developers, specialty societies, hospitals, health plans, persons and families etc.)

through:

• developing a monthly MMS Newsletter that will be distributed to the MMS Listserv by CMS staff through GovDelivery,

• updating the website with enduring materials,

• developing materials that compliment the Blueprint (ex. handouts) to help make the information accessible to less experienced measure developers, and

• other stakeholder engagement activities as requested by the COR.

The Contractor shall provide technical and administrative support for the maintenance of the contents of the CMS Measures Management System and other cms.gov websites associated with MMS work (ex. Pre-Rule Making, CMS Measures Inventory) by reviewing and updating the content, suggesting ways to improve communication of these sites to CMS and the public, and answering questions from stakeholders regarding material on the website. The Contractor shall develop a MMS stakeholder engagement and outreach strategy and update it annually.

Deliverable 8-15: Monthly MMS Newsletter Deliberable 8-16: MMS Stakeholder Engagement and Outreach Strategy

MACRA Focused Support and Education for CMS and Measure Developers:

The Contractor shall provide targeted outreach to measure developers interested in developing measures for the Quality Payment Program, including but not limited to specialty societies and patient advocacy groups. The Contractor shall provide a minimum of 12 Informational Sessions, http://www.qualityforum.org/ approximately monthly, on key processes of the MMS Blueprint to CMS staff, MIDS measure developers, and non-CMS contracted measure developers interested in developing measures for the Quality Payment Program by conducting WebEx that explain the key processes described within each document. The drafts are due 2 weeks prior to the session and final materials are due 1 week prior to the session. In addition to the 12 Informational Sessions, the Contractor shall host up to 2 public informational sessions via webinar to be open to the public (measure developers, clinicians, hospitals, etc.) for broad education on quality measurement and measure development as it relates to the Quality Payment Program.

The Contractor shall conduct outreach that includes newsletters/email distributions that are separate from other MMS activities and targeted for this audience. Newsletters and/or email distributions shall focus on advertising the Informational Sessions, as well as communicating other events, trainings, rulemaking deadlines, etc. that relate to CMS measure development work. The Contractor shall distribute such communications up to four times a month at the direction of CMS staff.

The Contractor shall address informational questions and inquires from measure developers interested in developing measures for the Quality Payment Program. Measure developers requiring additional assistance beyond information, such as support finding a data source or analyzing results, should be systematically tracked and refered to CMS staff or another designated entity.

Additionally, the Contractor shall facilitate up to 12 Spotlight sessions for the targeted audience to showcase their measures and measure development activites for CMS. The purpose of these showcase opportunities will be to provide a standing time on CMS staff schedules when stakeholders can present measures under development and garner feedback from CMS. The Contractor shall conduct outreach to solicit agenda items, get CMS approval one week prior to the meeting, provide an agenda and other meeting materials two business days before the meeting, and action items within two business days of the meeting.

Due to the separate funding source, the Contractor shall invoice and provide monthly progress reports on the activities described under MACRA Focused Support and Education for CMS and Measure Developers separately from all other tasks described in this TO SOW. When writing the technical and business proposals, the Contractor shall clearly separate the the activities described under MACRA Focused Support and Education for CMS and Measure Developers separately from all other tasks described in this TO SOW.

Deliverable 8-17: Monthly Information Session agenda, materials, and minutes Deliverable 8-18: Public Information Session agenda, materials, and minutes Deliverable 8-19: Spotlight session agenda, materials, and minutes

8D. Program, Measure, and Initiative Assessment:

CMS Measures Inventory Tool (CMIT): CMIT hosts the CMS Measures Inventory, along with other applications, in website form to promote transparency and accessibility for CMS, MIDS Contractors, and stakeholders (other federal agencies, specialty societies, patient advocacy groups, researchers, providers, facilities, and many more). The primary content of CMIT is outlined below, along with technical specifications about the system.

CMS Measures Inventory: The Contractor shall maintain and update the CMS Measures Inventory, designed to provide detailed information to stakeholders regarding the quality measures developed and/or used specifically by CMS. The CMS Measures Inventory provides CMS with the capacity to respond quickly to inquiries from internal and external stakeholders regarding such issues as measure types, uses, and utility of measures included in the CMS inventory. Measures include those that are under development, being considered through the pre-rulemaking process, and are proposed, finalized, or removed through the federal rulemaking process. The CMS Measures Inventory lists each measure by program, reporting measure specifications including, but not limited to, numerator, denominator, exclusion criteria, measure type, and NQF endorsement status. The goal of the CMS Measures Inventory is to provide a comprehensive list of measures that have ever been developed, considered, proposed, or utilized in a CMS program or initiative. The Contractor shall provide ongoing maintenance of the inventory, which include the following areas:

• Updates to the measures in the Inventory based on Federal Register notices, measure specification manuals, deliverables submitted by other MIDS contractors and information provided by CMS Programs

• Elicit validation by CMS program leads that measures are current and accurately represented in the Inventory

• Identification of new measures that are not included, and should be included in the Inventory

• Identification of CMS measures in the Inventory that should no longer be included and must be properly retired (withdrawn, moved to archive, or other dispensation).

• Clear display of the date of each update and regaular maintenance of a glossary with the definitions of fields and controlled vocabulary values, available to users.

Updates and refinement to supporting matierals (ex. user guide, FAQs)The Contractor shall update the CMS Measures Inventory at least twice a year, but up to quarterly per the guidance of the COR, in accordance with major CMS rule publications. A brief Summary Report highlighting updates shall be provided to CMS for posting on the CMS Measures Inventory Website as a public document concurrent with the Measures Inventory updates. The contractor will design a template for the Summary Report, prior to the first posting, with CMS review and approval of the template.

The Contractor may also be asked periodically by CMS to perform additional ad hoc analyses of the Inventory to identify gaps in the range of areas not included in the current Inventory, measures available in certain areas/topics, measures aligned in federal programs, and other topic areas identified by CMS.

The Contractor will prepare a listing of all CMS measures up to twice a year to update other prominent inventories of quality measures (ex. the HHS Measures Inventory) and other CMS partners working on CMS measures as directed by the COR. This technical and administrative support specifically includes preparing the list of measures in accordance with the requirements defined by the other measure inventory owner with guidance from the COR.

Deliverable 8-20: CMS Measures Inventory

CMS Measures Inventory Tool (CMIT): CMIT hosts the CMS Measures Inventory and ESST in website form.

The Contractor shall maintain and refine the two versions of CMIT:

• Public Verison: available to everyone and does not require a log-in. Features of the public facing version, located at cmit.cms.gov, include but are not limited to:

o Human-centered design display o Search, filter, and sort capabiltities in multiple formats to maximize flexibility o Measure details (ex. numerator, denominator, exclusion criteria, measure type, NQF endorsement status)

• Internal Version: includes all features of the public facing website with additional capabilities. The internal CMIT version requires a log-in. The users for the interal CMIT website include CMS, federal agencies, and CMS contractors. The internal CMIT website allows users access to advanced capabilities that include but are not limited to:

o ESST (described in 3.A. Environmental Scan) o Ability for individual users to save searches o Multiple users roles that can allow edit and review functionality by users other than the administrator

The Contractor shall maintain and refine the Health Systems Harms Dashboard (HSHD) application (described in 3.A. Environmental Scan). The Contractor shall integrate the HSHD into the internal version of CMIT, as directed by the COR.

This system does not require the Contractor the interface with other contractor’s systems, CMS systems, or other agency’s systems. The CMS Measures Inventory Tool has a current ATO that must be maintained as outlined in the Statement of Work. The Contractor shall also have the ability to conform with all CMS privacy and security standards, particularly treating all consumer data with protections inherent to government-to-consumer interactions. To fulfill these requirements, the Contractor shall enter into a data use agreement (DUA) with CMS to gain access to the CMS FISMA Controls Tracking System (CFACTS).

The Contractor shall be prepared to receive code and documentation for CMIT, ESST, and HSHD to build, deploy, and operate the systems in a new environment.

CMIT is composed of 3 layers, with each running on a different server. The data layer consists of two Microsoft SQL Server 2012 databases, one for storing the core of the measure inventory and one for storing the relational environmental scan data, and one GraphDB database for storing the semantic environmental scan data. The application layer consists of two Java web services, one for access to the inventory data and one for access to the environmental scan data, and one Apache Solr instance to provide full text indexing and search capabilities. The presentation layer consists of one Java web application built upon Apache Struts and Java Server Pages (JSP). A four environment model is required: DEV,TEST,IMPL,PROD. The user interacts with the web application over HTTPS (TLS) using a relatively modern browser which supports HTML5, ECMA/JavaScript, and CSS. All Java server components are currently hosted within Apache

Tomcat 8.5. All three layers currently run on Windows Server 2012. All application code is built using Java 1.8. CMIT is designed and tested to be compliant with the CMS Acceptable Risk Safeguards (ARS), version 3.1, including privacy controls, which subsume the FISMA NIST Moderate and Privacy controls.

The Contractor shall maintain all security guidelines and protocols required by HHS and CMS in order to maintain the system’s security compliance. This includes, but it not limited to maintaining an active ATO and complying with the lastest CMS ARS.

The Contractor shall maintain a user guide for the website to assist users in navigating both the content and software of CMIT. The Contractor shall in provide a mechanism for CMS, Contractor, and stakeholder feedback and system changes to be reviewed by CMS systemically to facilitate updates and refinements to CMIT.

Deliverable 8-21: CMIT Security Documentation (ATO) Deliverable 8-22: CMIT Roadmap (includes all applications) Deliverable 8-23: CMIT User Guide (includes all applications) Deliverable 8-24: CMIT Usage statistics (includes all applications)

CHAPTER 9. PUBLIC REPORTING/COMPARE SITES

CHAPTER 10. ACCESS TO SYSTEMS/DATA

In accordance with (IAW) Chapter 10 of the MIDS IDIQ SOW, the contractor shall carry out all work/tasks described in sections 10.A.

CHAPTER 11. QUALITY ASSURANCE AND PERFORMANCE

EVALUATION

In accordance with (IAW) Chapter 11 of the MIDS IDIQ SOW, the contractor shall carry out all work/tasks described in sections 11.A – 11.D.

CHAPTER 12. TRANSITION FROM INCUMBENT TO SUCCESSOR

In accordance with (IAW) Chapter 12 of the MIDS IDIQ SOW, the contractor shall carry out all work/tasks described in sections 12.A – 12.D.

Additional Requirements/More Specific Description:

12A. General

At the beginning of this contract, the contractor shall have two (2) weeks to review the transition plan provided by the Incumbent Contractor. During the two (2) weeks, the Successor Contractor shall collaborate with the Incumbent Contractor to hold any meetings necessary to ask questions and transfer information. The Successor Contractor shall provide the agenda and materials (i.e.

list of questions for the Incumbent Contractor) a day before. The Incumbent Contractor shall cooperate fully with the Successor Contractor, as directed by the COR, to ensure that all services continue without interruption.

Deliverable 12-2: Transition Plan agenda and materials.