D.31 PM REPORTING AND ANALYTICS GUIDE.pdf

Department of Veterans Affairs

Reporting, Analytics, Performance Improvement & Development

Performance Measurement

External Peer Review Program Guide

D.32 ATTACHMENT 36C25622R0004

P E R F O R M A N C E M E A S U R E M E N T

External Peer Review Program Guide

Contact information:

Eva Aileen Sowinski RN, Dr. Law, CPHQ

EPRP/ORYX COR/ Measurement Coordinator Reporting, Analytics, Performance Improvement & Development (RAPID)

Performance Measurement 10A8 Phone: 302-415-0115/ Email: eva.sowinski@va.gov

Version EPRP V v1 Sept 2018 mailto:eva.sowinski@va.gov

TABLE OF CONTENTS

Section I: An Overview

1. External Peer Review Program (EPRP)

2. Who’s Who and What Do They Do?

3. Let’s Get Started: Measure Development

4. Data/Data Security

Section II: The Process

5. Sampling

6. Abstraction

7. The Data Accountability Checklist (DAC)

8. The Exit Report and Exit Conference

9. Reconsiderations

Section III: The Data

10. Reports

11. Quality Control and Improvement

12. Dates and Timelines

Section IV: Pulling It All Together

13. How Can You Participate in the Program?

Appendices Appendix A: Key Websites

Appendix B: How to Request Access to Quality Insights VHA EPRP Secure Site

Appendix C: How to Stay In-Touch: Email Groups

Appendix D: Data Accountability Checklist (DAC) Agreement

Appendix E: External Peer Review Program (EPRP) Document Guide

Appendix F: Policies

Appendix G: Glossary

Section I: An Overview

1. External Peer Review Program (EPRP)

2. Who’s Who and What Do They Do?

3. Let’s Get Started: Measure Development

4. Data/Data Security

External Peer Review Program

All things are difficult before they become easy. — Unknown his section will provide an introduction and overview of the External Peer Review Program.

For more than twenty-five years, the External Peer Review Program (EPRP) has been the key source of information documenting Veterans Health Administration (VHA) medical facilities’ high quality of care to the nation’s Veterans. EPRP is a critical part of a nationwide audit and feedback system, helping to drive improved quality performance by informing care providers and managers of specific opportunities for improvement and demonstrating improvement as it occurs.

VHA uses EPRP data for management, research and public reporting (e.g., Access and Quality in VA Healthcare; Centers for Medicare and Medicaid Services (CMS) Hospital Compare) (see Appendix A for Key Websites referenced throughout this document). VHA websites allow Veterans and members of the public to compare quality performance among VHA facilities and between VHA and private health care providers. Quality Insights, Inc. (formerly West Virginia Medical Institute (WVMI)) has partnered with VHA to develop and maintain this data source for system-wide quality management, as well as to ensure that the information provided is an accurate and complete “gold standard” for clinical quality and patient safety assessment. The data collection system is continuously monitored so that it accurately depicts the care Veterans receive. The quality control processes in place allow assessment of consistency and completeness of the information collected from VHA medical records and databases.

Data abstraction via the EPRP is detailed and guided by a contract between the VHA and the Quality Insights. Data abstraction is completed both remotely and on site at the various VHA Medical Centers (VAMCs) by credentialed individuals (abstractors) under contract with Quality Insights to provide this service.

VHA Medical Centers designate an EPRP liaison to assist with orienting the abstractor to the various documentation sources that are VAMC specific and with facilitating the data retrieval process. The liaison may secure proof of documentation of care through non-CPRS methods in order to enhance complete data review; but, the information must be part of the medical record.

The liaison works on behalf of the VHA in general (and, specifically, the VAMC and VISN) to ensure that the abstractor has exhausted all available documentation sources.

T https://www.accesstocare.va.gov/ https://www.accesstocare.va.gov/ https://www.medicare.gov/hospitalcompare/Data/About.html

The Process can be divided into 3 phases:

1 Specification Phase. Measure specifications originate from external organizations such as Centers for Medicare and Medicaid (CMS), The Joint Commission (TJC), National Committee for Quality Assurance’s (NCQA) Healthcare Effectiveness and Data Information Set (HEDIS) as well as internally from VHA Program Offices such as the Offices of Mental Health and Community Care. Once a performance measure is approved, data collection questions (DCQ) are developed to guide what specific data is desired from the medical record. The Performance Measurement team works with the Program Offices, Lead Committees or Boards, other subject matter experts and the EPRP contractor, Quality Insights, to identify those data points that are needed to fully measure a particular concept.

When the DCQ have been fully analyzed for inclusion, step 2 is implemented. The finalized DCQ can be viewed by quarter on the Quality Insights web application:

https://secure.wvmi.org/EPRP_DBQ/.

2 Software Design and Development Phase. The DCQ developed by Quality Insights and approved by VHA Performance Measurement team are then integrated into a software package that results in an ‘instrument’ to ensure the abstractors collect the data with consistent definitions and rules. When the software has been structured in such a way that all data being collected meets the intent of VHA and the scoring of the data answers the performance question, this phase is complete. Measures are reviewed at least quarterly for potential changes. However, software changes are made throughout the fiscal year if program weaknesses are identified and as measure specifications need clarification.

3 Abstraction Phase. This phase is divided into the abstraction, Data Accountability Checklist (DAC) Review, and Exit Conference. Quality Insights Abstractors are provided a list of cases for review (Pull List). With the aid of the data collection software, they review each medical record either remotely or onsite. Once the records are reviewed, those that did not meet measurement requirements are identified and presented in the form of a DAC.

VAMCs are provided a maximum of four days to review the DACs and provide any additional information supporting that the record meets the measure requirements. Once the review is completed, the Abstractor and Liaison present findings to VAMC leadership and staff during an Exit Conference. This information contains the finalized results of the abstraction process and is forwarded for posting on the Performance Measurement databases.

https://secure.wvmi.org/EPRP_DBQ/

Who’s Who and What Do They Do?

Take Aways

EPRP is a complex process involving many people across many organizations

Primary staff are based in Performance Measurement and Quality Insights

VHA facility Liaisons link the process to facility day–to–day operations

Facilities identify a primary and secondary liaison

Keeping in touch- email groups, Sharepoints… s you can see, the EPRP process involves multiple steps and people. If you think about the 120+ VHA Medical Centers and the task of abstracting data from hundreds of thousands of medical records each year, one can truly appreciate the level of coordination and teamwork that is needed. This team is composed of many people both within VHA and Quality Insights. Within VHA are Quality Managers and their staff, dedicated EPRP Liaisons, Performance Measurement and Analytical staff, all responsible for the smooth abstraction, analysis, and reporting of data. Within Quality Insights are parallel staff (mainly the Abstractors), Regional Managers, Analytical and Development staff.

Organizationally, Performance Measurement is within Office of Reporting, Analytics, Performance Improvement and Deployment (RAPID) and reports to the Offices of the Under Secretary and Principal Deputy Under Secretary for Health. Performance Measurement provides resources and reports supporting clinicians, managers and others to improve care to Veterans. Quality Insights is contracted to provide data collection, analysis and reports to VHA to monitor care to Veterans.

The Program is authorized by the EPRP Contract that is administered through Performance Measurement. This VHA–wide contract provides oversite, funding, monitoring, and evaluation of the Program. Performance Measurement also establishes abstractor master accounts and ensures annual training is completed. Facilities provide the assigned abstractor with local Computerized Patient Record System (CPRS) access and a place to work while onsite.

The reference chart below provides a quick look at individual role responsibilities; detailed descriptions follow. [Note: PM refers to Performance Measurement Team]

A

VHA Staff Quality Insights Staff Role

COR/Program Manager (PM) Program Director Program oversight

Measure Coordinators (PM) Development Group Measure development and monitoring

Subject Matter Experts (Program Offices)

Supports/drives measure development

Analytics Team (PM) Analytics Group Sampling/Data/Reports

IT Team Maintain Quality Insights website access

Quality Manager (VISN/Facility) Facility quality oversite

Facility Liaison (Facility) Abstractor Review/Exit Process

Regional Manager Consultation role

Field Director Oversight

Below is a list of roles and their specific responsibilities:

Abstractor (Quality Insights)

• Scheduling on-site visits for review and Exit conferences with facility

• Reviewing medical records

• Reviewing DACs with facility Liaison and make any changes as needed

• Participating in Exit conference, providing any explanations or answering questions

• Providing general guidance regarding the review process

Contracting Office Representative (VHA COR)

• Providing oversight of EPRP process and managing contract process

• Serving as contract point of contact and VHA side supervisor

EPRP Liaison (VHA facility based)

• Requesting/maintaining access to Quality Insights VHA EPRP Secure site (see Appendix B)

• Orienting the Abstractor to the VAMC staff and process

• Assisting in gaining local computer access

• Assisting in providing medical record information not found in CPRS

• Scheduling with Abstractor any onsite visits as needed

• Reviewing DACs within the 4-day timeframe and providing any missing information as found to the Abstractor for any changes

• Organizing and overseeing Exit conference

VHA Facility Quality Manager

• Providing oversight of local EPRP process

• Participating in monthly Exit conferences

• Providing guidance as needed

Quality Insights Regional Manager

• Provides general information and updates regarding the process

• Is the point of contact for the Abstractor and the next step in the chain for any questions for the EPRP Liaison

• Helps resolve disagreements between Abstractor and VAMC

• Provides quarterly review as part of Quality Improvement process

• Assist back-up abstraction as needed

• Participates in training, special study abstraction, and assist as needed

Quality Insights Program Director

• Provides oversight of program

• Works with VHA to identify and develop measures

• Reviews initial measurement results

• Monitors measurement data

• Provides feedback to Program Offices

• Serves as measurement consultant and point of contact for VHA COR

Quality Insights Director of Field Operations

• Manages field abstraction process and staff

• Provides training and orientation to new Abstractors

• Participates in all aspects of EPRP process

• Assists in resolution of any issues regarding abstraction

• Provides feedback to and from field staff

Quality Insights Development Team

• Assists Performance Measurement staff in development of DCQ and guidelines directing measurement

• Meets regularly to discuss changes, review new tools, and plan

• Program questions and make any edits as required

Quality Insights Analytic Team

• Notifies Quality Insights VHA Project Manager of any problems affecting EPRP

• Provides quality analysis

Quality Insights Website Helpdesk

• Provides assistance with any problems to users of Quality Insights website

VHA Program Manager (EPRP)

• Provides program oversight

• Works with VHA Program Offices and other Groups to identify measures

• Develops measurement specifications

• Pilots and focuses review of initial measurement results

• Monitors measurement data

• Provides feedback to Program Offices

• Serves as measurement consultant and point of contact for EPRP

VHA Performance Measurement

• Serves as liaison to Program Office

• Works with Quality Insights to develop VHA measure specifications

• Validates and monitors measure data

• Maintains e-Technical Manual and corresponding communications

• Provides data management, analysis and reporting

• Plans sampling of medical records

• Provides statistical support and monthly sampling for review

A note about Abstractors:

EPRP Abstractors are Registered Nurses (RN), Registered Health Information Administrators (RHIA), or Registered Health Information Technicians (RHIT). Initial training consists of a one week “boot camp”. At the conclusion of the one-week session, abstractors are required to attain a minimum of 93% agreement rate on a gold standard abstraction. Following initial training, one-on-one field training with a Regional Manager includes where to find relevant documentation in CPRS and the DAC reconciliation process. Additional training is conducted by conference calls which include presentation of additional data collection instruments. New abstractors are closely monitored by the Regional Manager during the first several quarters of review.

Solving problems:

While the EPRP process typically runs smoothly, issues do arise periodically. If the Abstractor and facility Liaison are unable to resolve an issue at the facility level with assistance from the Regional Manager, the issue is forwarded to VHA COR/PM for a final decision. Decisions are a result of discussions between the Quality Insight’s Field Director, VHA COR/PM and other staff as appropriate (Measure Coordinators, Program Office, TJC, etc.). The final decision is made and communicated by the VHA COR/PM.

How to Stay In-Touch: Email Groups (see Appendix C)

Performance Measurement provides updates, education, and announcements in a variety of ways. The primary way is through email groups. There is a general email group that focuses on training announcements, general updates and information (VHA CO PMCC). Anyone can request to be added to that group. For information or requests to be added to the PMCC email group please contact Patricia Kearney at patricia.kearney@va.gov.

There is a specific group for EPRP liaisons only (VHA CO EPRP). Liaisons should contact Patricia Kearney at patricia.kearney@va.gov to be added or deleted to this EPRP Liaison Mailgroup.

•VHA Performance Measurement provides Sample (Pull List) to Quality Insights

•Quality Insights distributes to facilities via secure website

Abstraction

•DAC Review

•Any questions, issues, negative trends are resolved or brought to the Regional Mananger

Exit •Any unresolved questions/issues need to go through the Reconsideration Process

•Any remaining unresolved issues are referred to VHA COR/PM for review and final decision

Decision

Communication of questions/issues:

mailto:patricia.kearney@va.gov

Let’s Get Started: Measure Development

Measures come from external and internal sources

Measure specifications are developed into instruments and algorithms

Instruments are complex electronic forms that guide Abstractors through the data collection process

Scoring Algorithms convert raw clinical data into useable data easure development specifications originate from numerous sources such as internal VHA Program Offices or external regulatory agencies such as CMS/TJC. Measures are developed often in response to a question, to monitor a new process or program, or to help VHA compare care to outside organizations.

The measure development process includes:

• Identifying specifications for selected measures

• Development of the instrument

• Development of the algorithm

• Testing and validating

The EPRP Data Collection Questions are grouped into Instruments that are developed by the Performance Measurement team with the sponsoring group and Quality Insights. The Abstractors use these Instruments provided in a software package to gather data.

These Instruments are complex electronic forms that guide Abstractors through the collection of clinical data and forward the abstracted data to a central database for storage, analysis, and reporting. An Instrument consists of a set of questions and describes logic to enter responses to each question and determine the availability and sequence of questions necessary for accurate data collection.

M

Just some terminology clarification:

Instrument - Software (data collection question in electronic form) containing question specifications including administrative fields, responses, logic flow, and definitions/decision rules to guide abstraction of the record. When Instruments become too large they are sometimes broken into Modules (groups of questions related to one area within the Instrument).

Scoring Algorithm - pictorial representation of how the measure is scored, identifying scored data elements.

Instrument

Data Collection Questions

Scoring Algorithm

Single Instrument

(ex: CGPI)

Module

(ex: MH)

Data Collection Questions

Scoring Algorithm

Module

(ex: PI)

Data Collection Questions

Scoring Algorithm

Module

(ex: Core)

Data Collection Questions

Scoring Algorithm

Instrument with Multiple Modules

Below is an example:

VHA EPRP Clinical Practice Guideline and Prevention Indicators (CGPI)

Prevention Indicators Module

Name Question Field Format Definition/Decision Rules

18 tobscrd t

Enter the most recent date within the past year that the patient was screened for tobacco use.

mm/dd/yyyy

< = 1 year prior to or = stdybeg and < = stdyend

If notobuse = 1, will be auto-filled as 99/99/999 Abstractor may enter 99/99/9999 if the patient was not screened within the past year

If 99/99/9999, auto-fill tobnow as 95, tobuseyr as 95 tuconsel as 95, tucnsldt as 99/99/9999, tucrefer as 95, tucrefdt as 99/99/9999, offtucrx as 95, tucmedt as 99/99/9999, ptreqrx as 95, and go to colondx as applicable

Review ALL notes and enter date of most recent screening.

Most recent date: the date most immediately prior to or during the study interval when the patient was asked whether he/she was a current tobacco user. May be by direct question to the patient or completion of a patient questionnaire form.

If the patient was not screened for tobacco use within the past year, enter default date 99/99/9999.

Most recent date may be taken from either the inpatient or outpatient record. Date must be specific. Use of default 01 is not acceptable.

Question

Question Responses

Definitions /

Decision Rules

Question number

Corresponds to question on algorithm

Question mnemonic

Scoring Algorithms convert a collection of raw clinical data elements into useable data. The Scoring Algorithms are in a flowchart format, representing the steps of measure calculation pictorially. Each Scoring Algorithm is a schematic that shows how to generate denominators and numerators to score the measure. Performance Measurement communicates question and measure changes prior to the start of each new quarter. Quality Insights reviews these changes as well as those applicable from external comparator systems to which VHA reports and modifies questions and measure calculation algorithms accordingly. These changes are reviewed by Performance Measurement and Quality Insights a second time for accuracy and consistency.

Below is an example of a Scoring Algorithm:

Both Instruments and Scoring Algorithms are updated, tested, and validated quarterly to ensure consistent and accurate review.

The elements of the Instrument and their interrelationships are shown in greater detail in the Abstraction section of this manual.

Q18 from shown on previous page

Data / Data Security

Federal Regulations are followed to ensure data confidentiality and quality assurance

Data is used for performance measurement and improvement

Data is transmitted daily, monthly, and quarterly through secure tunnel ata security is critical in today’s virtual world. EPRP ensures confidentiality of both sensitive patient medical, patient identifiable, and quality assurance data in accordance with Federal Regulations.

The data collected provides facilities with an immediate report of findings; this includes a daily uploading of data at the completion of the facilities’ monthly review (Exit Conference).

The data (raw and scored) is transmitted to Performance Measurement through a secure tunnel daily, monthly, and quarterly, as scheduled.

The data collected through the EPRP process is used for internal and external monitoring and reporting. Performance Measurement maintains a variety of useful reports that are discussed in detail later in this manual (see Section III.10).

A schematic of the data transmission process is pictured on the following page.

D

Section II: The Process

5. Sampling

6. Abstraction

7. The Data Accountability Checklist (DAC)

8. The Exit Report and Exit Conference

9. Reconsiderations

Sampling

Sampling is conducted by a Statistician/Program Analyst within Performance Measurement

Sampling is categorized into Outpatient and Inpatient ampling is a critical step in the EPRP process. It is based upon sound statistical principles, and sizes and methods may vary based upon specific Catnums (i.e., the category number used during EPRP to identify patient cohorts for sampling). Sampling is conducted by a Statistician/Program Analyst within Performance Measurement. The resulting samples (called Pull Lists) are then provided to Quality Insights for distribution, abstraction, and scoring.

This chapter provides a brief overview of both the outpatient and inpatient sampling processes.

Full details can be found in the EPRP Sampling and Catnum and Cohorts documents located on the External Peer Review Program (EPRP) Resources website. And training on these and other EPRP topics can be found on the Performance Measure Training page.

Outpatient Sampling

The EPRP outpatient sampling design is based on several working principles:

• Collect clinical data on a random sample of established VHA users;

• Maintain the ability to generate valid estimates at the VISN level;

• Maintain the ability to generate valid estimates at the facility level for the greatest number of quality measures;

• Use available financial resources to efficiently distribute cases across the VHA system for as many measures as possible, and;

• Report annual facility estimates within approximately +/- 6 points of the true facility rate (based on 95% confidence intervals).

In general, a fixed number of NEXUS (i.e., the outpatient clinic stop codes that most accurately identify extensive users of the VHA system) cases are randomly sampled from each health

S http://vaww.car.rtp.med.va.gov/programs/pm/pmEPRP.aspx http://vaww.car.rtp.med.va.gov/programs/pm/pmLearning.aspx facility (Sta3n) regardless of the population available. There is some variation in the fixed allocation across VHA facilities, but the allocation is similar each month. Additional EPRP outpatient goals were initiated in FY11, and were designed to improve sampling. They included:

1) link patients to a facility where most of their primary care (PC) services were obtained in the prior year, 2) fully utilize augmented data in scoring of all outpatient measures, and 3) increase the proportion of patients sampled from CBOCs.

Example from CGPI Exit Report:

Cases are selected from a specific time period designated as the “study interval.” To qualify for outpatient abstraction, the patient must have been seen by an MD, DO, APN, PA, or Psychologist in one of the NEXUS clinics within twelve months from the first day of the study interval. All cases selected for review must also have had another visit anywhere in VHA within the past two years. The “NEXUS clinics” include primary care and specialty clinics as well as selected mental health clinics. Although all applicable data is collected, mental health patients whose primary care is received in a non-VHA setting are excluded from VHA designated indicators.

Due to the inherent over-representation of cases from small sites and the under-representation of cases from large sites, cases are weighted based on patient characteristics such as gender, diabetes, and acute myocardial infarction (AMI) status. Weighting corrects for the unequal number of available cases within each organizational level (i.e., CBOC or facility) and protects against biased or inaccurate scores. This may lead to scores that do not seem mathematically correct (i.e., 8 out of 10 may not = 80%, but 67% or 87% depending on the weights of the cases that pass and fail the measure). Specifics about weighting are found in the EPRP Sampling document. Cases with the same patient characteristics at the same facility will have the same weight in a given sampling month.

The Joint Commission (TJC) ORYX Measure Sampling Requirements

Sample sizes for the TJC ORYX Measures are measure-specific. VHA performs monthly sampling and must ensure that its initial patient population and sample sizes adhere to the measure-specific requirements (e.g., for the Global Measures, populations less than 51 are not sampled, 100% of the patient population is required, but for populations 51-254 patients, a sample of 51 patients is required, etc.). Inpatient reviews usually occur 1-3 months after discharge when coded information has been transmitted to the Corporate Data Warehouse (CDW) by the facility and is identifiable for inclusion on the Pull List.

Monthly Initial Patient Population

(size “N”)

Minimum required sample size “n”

Great than or equal to 510 102

255-509 20% of Initial Patient Population

51-254 51

Less than 51 100% of initial Patient Populations required http://vaww.car.rtp.med.va.gov/programs/pm/pmEPRP.aspx

Pull List

As noted, the samples are referred to as Pull Lists in the EPRP process and are essentially considered the denominator in the scoring process. The Pull List contains the names of the patient records to be reviewed (abstracted) (see example below). Once Quality Insights receives the Pull Lists, the abstraction process begins. The following chapters will provide greater details about each of the subsequent steps in the EPRP process.

• The first Pull List of each quarter is pulled during the first full week of the first month of the quarter

• The second Pull List is pulled during the 1st full week of the second month of the quarter

• The third Pull List is pulled during the last week of the second month of the quarter

Please see Dates and Timelines in Section III.12 for a complete calendar and explanation.

Example of Pull List Downloaded from Secure Quality Insights Website (fictitious site)

Abstraction

Abstractors review medical records from a Pull List generated by VHA’s sampling process.

Data Collection Questions are designed to guide the Abstractor to collect data accurately

Only the official medical record can be used for abstraction.

The time period from Pull List release to the “Last Day to Exit” is approximately 30 days.

bstractors review medical records for data elements that meet the measure specifications. For example, veterans are expected to be screened for tobacco use. The abstracted data elements are scored using Algorithms that determine if cases pass or fail the related tobacco measures.

Performance Measurement produces lists of records to be reviewed, called Pull Lists (see Sampling section). EPRP Abstractors collect relevant information for patients on the Pull List using Data Collection Questions developed by Quality Insights in conjunction with Performance Measurement measure coordinators and subject matter experts. The abstracted data is entered into software and scored using Algorithms that convert the raw data into usable quality measures. Abstractors review medical records via remote access to the Computerized Patient Record System (CPRS). Remote data is accessed via Joint Legacy Viewer (JLV) when appropriate. Abstractors are notified by an email generated by Quality Insights that the Pull List is available for review and they can begin abstraction. Abstraction must be completed, data reconciled and an Exit Conference completed within approximately 30 days. Please see the EPRP Field Calendar for Pull List dates and the Last Day to Exit each Pull List. Only the official medical record may also be reviewed for relevant information (e.g. progress notes, laboratory reports, radiology reports, reports of procedures and discharge summaries).

Abstractors may not take documentation from Clinical Reminders or Mental Health Assistant (MHA) unless the source populates a signed progress note.

The Abstractor’s response to each Data Collection Question is guided by associated Definition/Decision rules. An example of a question and associated Definition/Decision Rules is below:

VHA EPRP Clinical Practice Guideline and Prevention Indicators (CGPI)

Prevention Indicators Module

Abstractors review all relevant medical record documentation. For example, when abstracting a CGPI record, the Abstractor reviews all notes for the study year and older notes as applicable.

For inpatient topics, the Abstractor reviews all notes for the admission specified on the Pull List.

The rules for some questions include guidance as to which data sources may be abstracted.

Suggested Data Sources serves as a guide for the Abstractor but does not limit them to only looking in those documents. ‘Only Acceptable Sources’ instruct the Abstractor to only collect data from specific sources.

For example:

ONLY ACCEPTABLE SOURCES: *Emergency Department record nursing unit admission assessment/admitting note; observation record; procedure notes (e.g., cardiac catheterization, endoscopies, surgical procedures)

Some rules include ‘Excluded Data Sources’ (i.e. data cannot be abstracted from these sources).

The Data Collection Questions software utilized by the Abstractor has many built in parameters to help ensure accurate abstraction. For example, date parameters ensure that only actions or

Name Question Field Format Definition/Decision Rules

18 tobscrdt Enter the most recent date within the past year that the patient was screened for tobacco use.

mm/dd/yyyy

< = 1 year prior to or

= stdybeg and < = stdyend

If notobuse = 1, will be auto-filled as

99/99/999

Abstractor may enter 99/99/9999 if the patient was not screened within the past year

If 99/99/9999, auto-fill tobnow as 95, tobuseyr as 95 tuconsel as 95, tucnsldt as

99/99/9999, tucrefer as 95, tucrefdt as

99/99/9999, offtucrx as 95, tucmedt as

99/99/9999, ptreqrx as 95, and go to colondx as applicable

Review ALL notes and enter date of most recent screening.

Most recent date: the date most immediately prior to or during the study interval when the patient was asked whether he/she was a current tobacco user. May be by direct question to the patient or completion of a patient questionnaire form.

If the patient was not screened for tobacco use within the past year, enter default date 99/99/9999.

Most recent date may be taken from either the inpatient or outpatient record. Date must be specific. Use of default 01 is not acceptable.

services provided on a date within the acceptable timeframe can be entered. There are also features that prevent the collection of conflicting data. Warnings let an abstractor know if a value entered may be outside of an expected range (e.g., a weight of 600 lbs.).

In addition to the abstraction guidance in the rules and the software parameters, the DACs provide facility staff an opportunity to find documentation that was overlooked or not abstracted during initial review. Below is a diagram of the Abstraction Process, which will be discussed in detail in the following section.

THE ABSTRACTION PROCESS

Abstraction Process (30 days)

Abstractors and Liaisons receive an email alert that the Pull Lists are available for review

Abstractors access Pull Lists and schedule the Exit Conference with Liaison within one week

Designated EPRP facility Liaisons access Pull Lists via Quality Insights secure website

Abstractors complete remote abstraction and export DACs to Quality Insights website per DAC agreement (see DAC section (Section II.7))

Move to Step 2 (DAC Section)

The Data Accountability Checklist (DAC)

Data Accountability Checklists (DACs) provide a list of all measures that fail for each case (patient) on the Pull List and allow an opportunity for the facility staff to find data that may have been missed or not considered appropriate by the Abstractor on initial review

DACs are available on the Quality Insights secure website in advance of the monthly Exit Conference.

The number of days that DACs are available prior to the Exit Conference (2-4 days) is determined by DAC Agreement negotiated between the Abstractor and facility Liaison. The agreement also notes the venue for the Exit Conference i.e. telephone, onsite, or a combination of telephone and onsite.

Prior to the Exit Conference, documentation found by the facility during review of DACs is considered by the Abstractor and data is changed when appropriate (DAC reconciliation).

he Data Accountability Checklists (DACs) are an important part of the EPRP abstraction process. DACs are generated for each case from the data the Abstractor enters and:

• Shows which measures do not pass or;

• Shows that all measures do pass; and

• Shows which cases are excluded from all measures.

DACs allow facility personnel to review records for documentation the Abstractor may have missed and that will allow measures to go from fail to pass or from fail to exclude.

The DAC process starts with a negotiated agreement between the liaison and abstractor (see Appendix D). The agreement determines how the Exit Conference will be held (onsite, by phone or a combination of phone and onsite) and how many days (2-4) in advance of the Exit Conference the DACs will be available for review. The current EPRP contract specifies onsite exit conferences.

However, with approval from the EPRP COR, facility staff may elect to have all or some of the Exit Conferences by telephone. The agreed upon number of onsite visits per quarter (1, 2, or 3) is stipulated in the DAC agreement. Performance Measurement requires a minimum of one onsite exit for each facility per year.

The DAC Agreement specifies the number of days in advance of the Exit Conference the facility will be able to access the DACs on the Quality Insights secure website. The number of days may not be greater than 4 nor less than 2 days and may be different for each measure set. For example, the number of days for CGPI DACs may be 4 and the number of days for VTE DACs may be 2.

The EPRP Abstractor is responsible for completing the DAC Agreement Form (see Appendix C) after an agreement has been reached and will provide a copy to the Liaison. A new DAC Agreement form should be completed when:

• There is a change in the EPRP abstractor or the liaison or back up liaison;

• A change in the number of DAC days or the how the Exit Conference is held is requested; and

• At the beginning of each new fiscal year.

The EPRP Abstractor will export the DACs to the Quality Insights secure website in accordance with the negotiated DAC Agreement (Appendix C): https://eprp.wvmi.org/Home.aspx. This process triggers an automated email message that alerts the Liaison that DACs are available. DACs may then be distributed to team members for review as per facility procedure. The Abstractor will also export a Preliminary Exit Report at the same time the DACs are exported. This report will provide a preview of measure scores prior to the DAC reconciliation process. The Preliminary Exit Report is also accessed via the secure website.

Following the review of DACs by facility personnel, the DACs are reconciled with the EPRP Abstractor. DAC reconciliation can be accomplished in several ways. If the Abstractor makes onsite visits, DAC reconciliation may be done at that time. DACs may also be reconciled by phone or via email exchange. It is important to remember that Abstractors do not have encrypted email so no Personal Health Information (PHI) can be included in the communication. The control number assigned to each case on the Pull List should be used to identify the patient. The Abstractor considers additional data located by facility staff and changes his/her data when appropriate. If the Abstractor does not feel a change in data is warranted, he/she will provide an explanation. The EPRP Regional Manager should be consulted when the Abstractor and Liaison do not agree on whether documentation produced by the Liaison is sufficient to change the Abstractor’s data. DAC reconciliation should be completed at least one hour prior to the Exit Conference in order to allow the Abstractor time to finalize data, export the Exit Reports and prepare for the Exit Conference.

Prior to the Exit Conference, the Abstractor will export Final DACs that will reflect any changes made to the data during the DAC reconciliation process. It is important to review Final DACs since changes to data may not affect the measure as anticipated. Changes to the data may also cause different measures to fail.

https://eprp.wvmi.org/Home.aspx

The control number is a unique identifier for each patient; can be used to ID patient instead of name or SSN

This indicates where patient receives most of care, facility, CBOC. You can determine what abstractor enters here.

After DAC reconciliation sa7 now passes the measure (i.e., is included in the numerator) and no longer appears in this section of the

DAC.

Measures not meeting:

pvc11h, sa7, pdst51, smg8.

After DAC reconciliation sa7 now passes the measure (i.e., is included in the numerator) and no longer appears in this section of the DAC. However, since the patient has screened positive for Alcohol Use, the follow-up measure related to counseling sa17 in now failing since counseling was not documented.

Facility Liaision receives notification that DACs are available on the website via automated email and email/phone call from Abstractor

Liaison reviews DACs and/or distributes DACs to facility stakeholders for review

Liaison and Abstractor agree on date/time and method (phone, email, onsite) for DAC reconciliation

DAC Reconciliation Process is completed 1 hour prior to Exit Conference

Abstractor exports Final DACs to EPRP secure website

Liaison reviews Final DACs

Move to Step 3

(Exit Conference (Section II.8))

THE DAC REVIEW PROCESS

The Exit Report and the Exit Conference

An EPRP Exit Report for each abstraction topic is sent to the secure website following DAC Reconciliation.

The Exit Report provides a summary of the outcome of monthly abstraction and includes the numerator, denominator and score for each measure.

Scoring Algorithms, Exit Report Guides, and Case Level Reports are available to assist with understanding the reports

A monthly EPRP Exit Conference is conducted either onsite or by telephone.

The Exit Conference must be scheduled in accordance with the “Last Day to Exit” as detailed on the EPRP calendar.

The EPRP Abstractor has references and knowledge that can help make the Exit Conference useful and interesting.

bstractors should visit the assigned VHA medical centers at least annually (and no more than monthly) after negotiating the visitation date with the assigned VHA EPRP Liaison. If more than one Abstractor is assigned to abstract at a facility, only one Abstractor will conduct the onsite visit. The date of the Exit Conference should be confirmed with the EPRP Abstractor as soon as possible after release of each Pull List. Quality Insights is required to make the dates of all Exit Conferences available to the VHA COR/PM one week before the Exit Conference (see The EPRP Field Calendar).

The onsite visit shall take place during normal business hours when visiting the facility unless otherwise agreed upon by the Liaison. Abstractors may not be on VHA premises without the Liaison or assigned back-up person present. Telephone, Web or Videoconferencing Exits may occur in lieu of onsite visit with the consent of the COR. The facility will provide office space, an analog phone line for access and access to CPRS.

The objectives of the onsite visit are:

• Review with the Abstractor any data elements not found during remote review (DAC review). The facility EPRP Liaison may provide relevant documentation from the official medical record missed in the offsite abstraction. These data will http://vaww.car.rtp.med.va.gov/programs/pm/pmETechMan.aspx be entered into the data collection instrument by the Abstractor during the onsite visit when appropriate (see DAC Section II.7).

• Present the facility-level findings to facility stakeholders, including senior leadership and QM. After completion of the DAC reconciliation process, the Abstractor will export a Final Exit Report for each topic to the Quality Insights secure website. The Exit Report includes aggregate data from the facility’s monthly review.

The following reports are available on the Quality Insights secure website prior to the Exit Conference.

The Statistical Report shows the disposition of the records: reviews completed, excepted (do not review), blank, and any records in error status.

The Do Not Review Report lists (by control number) the records that were excluded by topic Validation. This report provides a brief explanation of why the record was not included in the review.

There is an Exit Report for each review topic that provides the numerator, denominator and score for each EPRP measure:

• A Preliminary Exit Report is available when the DACs are exported.

• The Final Exit Report is available after DACs have been reconciled and the Abstractor has finalized the data.

At the top of each Exit Report there is a summary of what is included in the report as well as information about how cases are selected for review. Additional information includes: the Pull List date, the study interval and the total number of records abstracted.

Column headers in the body of the Exit Report:

1. Mnemonic: short name for the measure, for example, ips1a or c9h.

– The mnemonic appears in the Indicator column of the Exit Report.

2. Measure description, for example

– Nutrition/hydration assessment by registered dietician within 30 days

– HTN and DM: BP < 140/90 age 60 – 85

3. The Numerator (NUM) is the number of cases that “pass” the measure.

– A DAC is created for each case that is not included in the Numerator, indicating what element was failed.

– For some measures it is not a good thing to be included in the Numerator, for example: HTN: BP > = 160/100 or not recorded.

– A down arrow beside the measure description indicates a lower score is better ( ).

4. The Denominator (DEN) includes all the cases that are scored for each indicator. The Denominator does not always equal the total number of records reviewed since some cases will be excluded from the Denominator by the Scoring Algorithm.

5. What is represented by the number in the Score column depends on the measure. In many cases it is simply the Numerator divided by the Denominator.

• On the CGPI report, the scores are weighted (see Sampling Section II.5)

– Verification of the weighted scores is possible using the Combined Measure Master Report found on Performance Measurement page.

• In some cases, the score does not represent a numerator and denominator.

– For example, the “score” for ed1 is the median time from ED arrival to ED departure for admitted ED patients.

6. The Exit Reports display

a. the scores for the current Pull List (column 5)

b. the quarter to date score

c. the cumulative score for the previous quarter

– These three columns can help you track progress over time as you review the report.

• Measures that appear in bold print on the report are considered Accountability measures and eligible for reconsideration.

• The Joint Commission Core Measures are italicized (bolded if used for Accountability).

• Pilot Indicators generally appear in a section separate from other quality indicators and/or are labeled as such.

• Measure scores are generated by Scoring Algorithms (not by EPRP Abstractors).

• Links to the Scoring Algorithm are in the Performance Measurement eTechnical Manual (eTM).

– The Scoring Algorithms are also on the Quality Insights website (no password required to access).

• An Exit Report Guide for each topic is also available on the Quality Insights website that is a “plain language” version of the Scoring Algorithms.

– The Exit Report Guide is a guide to interpreting the scoring, not a guide to abstraction.

• Case Level Reports are available on the EPRP secure website (password protected)

– These reports essentially “drill down” through the cases included in a measure and show how each case scored and the reason for fallout, when applicable.

– A tutorial is available on the website to guide you through the use of this feature.

Exit Conference

The EPRP Abstractor will work with the facility Liaison to set a date and time for an Exit Conference to be held at the conclusion of the monthly Pull List review. Things to consider when scheduling:

• The “Last Day to Exit” per the EPRP calendar

• Number of days for DAC review per the DAC Agreement

Deviation from scheduled review/Exit: Occasionally, something may interfere with the planned onsite visit and/or Exit Conference such as The Joint Commission survey or Office of Inspector General (OIG) visit.

It is important to notify the VHA COR/PM as quickly as possible to discuss rescheduling the Exit Conference with respect to EPRP deadlines.

EPRP Liaisons are generally responsible for arranging the Exit Conference, e.g.,

• Inviting attendees

• Arranging for the meeting room

• Setting up a VANTs line or Skype call if some attendees will join remotely

If printed copies of the Exit Reports are desired, it is the Abstractors responsibility to print the reports and make copies.

The EPRP Exit Conference should be designed to meet the facility’s needs. The EPRP Liaison determines the format of the Exit Conference but the EPRP Abstractor should have some suggestions about what might be helpful and of interest. EPRP Abstractors are trained to do more than just read the numbers on the report. For example, at the first Exit Conference of each quarter, the Abstractor can provide an overview of the changes effective for the current quarter, such as:

• Changes to existing questions and rules

• New measures

• Deleted measures

• Changes to measure Scoring Algorithm

• The Abstractors can also discuss things like

– Precisely what information the questions are seeking

– Why the documentation doesn’t meet the intent of the questions they must answer

– How documentation could be improved to meet the intent of the questions

– What is included in the numerator and denominator of the measures

Work with the Abstractor to be sure you get the most benefit from the Exit Conference. Once the Exit Conference has started, the Abstractor cannot make any changes to the data. If the Liaison feels a change is needed, they will need to follow the process for requesting reconsideration (see Section

II.9).

The date/time for the Exit Conference is determined within one week of Pull List release

Liaison secures meeting room for Exit Conference and invites stakeholders, sets up remote communication, etc.

Abstractor finalizes data and exports Exit Reports to secure website

Abstractor prints Exit Reports and makes copies for Exit Conference attendees

Abstractor conducts EPRP Exit Conference

THE EXIT CONFERENCE PROCESS

Reconsiderations

Reconsideration may be requested if additional supporting documentation is found after the Exit Conference or when the facility disagrees with the data the Abstractor has entered.

Measures eligible for reconsideration are in bold print on the Exit Reports.

Requests for Reconsideration are sent to the EPRP COR and must be sent within two working days following the Exit Conference. The timeline becomes narrower in the last month of each quarter. (See EPRP Filed Calendar).

If accepted for reconsideration, supporting documentation for the case is sent to Quality Insights following the process outlined above.

Quality Insights notifies the facility Liaison of the reconsideration determination via email.

s detailed earlier in Section II: The Process, the DAC Reconciliation process allows facility personnel a chance to present documentation to the Abstractor that may have been missed or deemed inappropriate for inclusion in the data during remote review. If facility personnel and the Abstractor do not agree on whether data is appropriate for inclusion and/or the facility finds additional documentation after the Exit Conference, reconsideration may be requested. It should be noted that the Abstractor cannot change the data once the Exit Conference has started.

Process for Reconsideration of EPRP Reports and findings:

• Only Accountability measures are eligible for reconsideration and are in bold print on the EPRP Exit Reports.

• There are no reconsiderations of Quality Indicators or Pilot Measures.

• All initial requests must be e-mailed to the EPRP COR via Outlook (Eva.Sowinski@va.gov) within 2 working days following the Exit Conference.

– This e-mail must include:

o Name and Number of the facility requesting reconsideration o Indicator mnemonic and descriptor o Contact name and phone number o Pull List type and Date o Exit Report date in which the case was reviewed o Short synopsis of the issue about which reconsideration is requested

No patient specific data may be forwarded over Outlook and is not needed at this stage of reconsideration. An initial review is completed by VHA COR/PM. If it is readily apparent that reconsideration does not change the outcome or if the request is past two working days of the Exit conference, the case is not accepted for reconsideration. Notification to this effect is made via email.

Note: E-mail requests for reconsideration that contain patient specific information are not accepted.