Appendix_B_-_Sampling_and_Extrapolation_Guidance.docx

TABLE OF CONTENTS

DCPA Guidance on Medical Review Sampling and Extrapolation

January 2015 - 3 - Introduction

This document is intended to provide detail and guidance regarding the sampling and extrapolation procedures outlined in the Medical Review Contractor (MRC) Statement of Work (SOW) for the review of cost-based Managed Care Organizations (MCOs). Of particular importance is the guidance from CMS regarding stratification, a threshold for materiality of errors in medical expense and statistics included in the cost report, and extrapolation calculations.

The MRC should assess the Plan’s care delivery model and data elements used for apportionment. The understanding obtained should be considered in applying the general methods described in this guidance. This document assumes that the MRC has received a clean, reconciled claims database that ties to the cost report Worksheet K from the Cost Audit Contractor (CAC). Additionally, this document assumes that the CAC has verified that the population from which the sample will be selected is complete and correct.

Stratification

The purpose of stratification is to reduce the possible variation in detected error amounts within a stratum by grouping similar sampled items. The result of reducing variation in the stratum error amounts is to reduce the overall variance in the sample and increase the resulting precision of the estimate. The MRC shall design a stratification plan that groups similar paid amounts.

CMS recommends that the MRC use between 3 and 5 strata. More strata could be used only if, in the Contractor’s professional judgment, additional strata would be beneficial for a specific reason. The MRC should document consideration of the number of strata in the sampling plan.

CMS recommends that the MRC first stratify the population by cost center. The Contractor should then examine the variation of claims amounts within each cost center to determine whether certain cost centers should be further stratified by dollar thresholds. Each Plan will be different, but typical cost centers to consider include Individual Physicians, Clinics, and Durable Medical Equipment. For cost centers that have wide variations in individual claims costs, stratification by dollar amount would help to lower the variation and improve precision. In addition, cost centers that have high-dollar claims can benefit from dollar amount stratification to limit the effect of a high-dollar error on the rest of the population. Zero-paid claims and refunds should be segregated in a separate stratum that is not included as part of the sample.

Not all cost centers may require further stratification. If the claims dollars are not substantially varied within the cost center, then further stratification may not be beneficial. The Contractor may then choose to leave the cost center as one stratum or even combine small cost centers where the payment amounts are similar. All of the Contractor’s rationale and considerations for the sampling methodology should be sufficiently documented in the sampling plan, including the determination of whether stratification of certain cost centers was deemed appropriate.

In order to retain the statistical properties of the sample within each stratum, the Contractor must select a minimum of 30 items per stratum.

Splitting Strata to Isolate Issues

If upon testing the Contractor finds a common error that is not systematic or predictable enough to exactly tabulate, the Contractor may consider the appropriateness of splitting that stratum to further isolate the issue. For example, perhaps a stratum contains injectible drugs as well as surgeries and other services. During the review, the Contractor observes that the providers are sometimes billing the incorrect units for some injectibles.

Specifically, for example, the Contractor sees that the provider is often billing 6 units for the 6 ml contents of one vial when they should bill 1 because the price is per vial. These billing errors result in the provider being paid multiples of the correct payment (6 instead of 1, in this example).

In the above example, the stratum could be split into injectibles and all other services in order to isolate the errors and improve overall precision. Splitting the stratum has the added benefit of reducing a plan appeal of these errors by not extrapolating the injectible errors to other types of services.

The minimum number of sample items per stratum would still apply, so each sub-stratum would still need to contain at least 30 sample items. This could require selecting additional samples (beyond the 350 minimum).

Determination of Individual Error Amounts

In a fee-for-service or fee-equivalent model, the expense amounts for services vary. The error amount determined during testing for a given service or claim can be the full value of the service/claim or a partial amount. If the service/claim is a duplicate payment or records cannot be found, for example, the value of the error is the full value of the sampling unit (service/claim). Thus, the correct value would be zero.

An example of an error for less than the full value of the sampling unit occurs when the service has been up-coded by the provider. In this example, the medical record supports a lower-level code than the code that was billed and paid. The value of the error (“difference amount” in RAT-STATS) is the difference in value between the service that was paid and the service supported by the medical record. In these circumstances, completely disallowing the code is inappropriate, but reducing the code to the code supported by the medical record is necessary.

Calculation of Financial Error Rate

The Contractor shall calculate an overall financial error rate for the Medicare sample tested. The overall financial error rate will be used to determine whether the error is material to the cost report based on the materiality threshold described in the next section. This calculation is only to determine if extrapolation will be necessary, but not for the purpose of calculating the extrapolation adjustments.

The overall financial error rate should be determined on both the statistic (column 3 of Worksheet K) and the reported total costs (column 6 of Worksheet K). It should be noted that the method shown below assumes a statistic based on paid dollars.

This same concept should be applied, when appropriate, for alternative statistics resulting in separate calculations for claims expense/costs and statistics. When uncertain, contractors shall refer to their Servicing Auditor for specific guidance. This method is used in the attached extrapolation template (provided by CMS).

1. An overall financial error rate will be calculated for each year of the Medicare medical expense samples tested.

2. If the sample is not stratified, the financial error rate is calculated as:

Total Sample Paid Dollars

3. If the sample is stratified, determine the weighted financial error rate, weighted by paid dollars, across the strata:

Materiality Threshold

The AICPA offers the following guidance in testing and determining materiality[footnoteRef:1], supporting CMS’ approach to testing and applying materiality standards to the Medicare sample results: [1: AT Section 101.67.]

“The practitioner should consider the concept of materiality in applying this standard [Attest Engagements]. In expressing a conclusion, the practitioner should consider an omission or a misstatement to be material if the omission or misstatement— such that a reasonable person would be influenced by the omission or misstatement. The practitioner should consider both qualitative and quantitative aspects of omissions and misstatements.”

Based on the calculation in the prior section, do not proceed to extrapolation if the total overall financial error rate across all strata for the year is less than or equal to 1%. The statistic and cost total overall financial error rates should be determined independently for each year.

If the total financial error rate across all strata in either cost, statistics or both is greater than 1%, proceed to extrapolation for both cost and statistics for the year. The basis for the 1% threshold is common industry standards, including public health texts and other published industry sources[footnoteRef:2]. [2: Including: Peter R. Kongstvedt, MD. 1996. The Managed Health Care Handbook, p.515. Aspen Publishers, Inc. and Blue Cross and Blue Shield Association. 2012. Member Touchpoint Measures.

Extrapolation

If the overall financial error rate for the population exceeds the materiality threshold described above, the Contractor should estimate the total error in the population through extrapolation. The Contractor should calculate the estimated lower bound for each stratum at 90% confidence using RAT-STATS and the strata information from the extrapolation template (as provided by CMS).

In RAT-STATS, if the plan measures expense on a fee-for-service basis or fee-for-service equivalent, the Stratified Variable Appraisal module would typically be the application for calculating the lower bound. Note that the input file for RAT-STATS must be Excel 97 if using an Excel format.

Use the lower bound to calculate any necessary adjustments to the medical cost and apportionment statistic. If the calculated lower bound is less than the total of the dollar errors detected in the sample, then the auditor should use the actual sample error.

The Contractor should calculate the lower bound results for each stratum at the cost center level. If the Contractor has stratified a cost center further by dollar thresholds, then calculate the lower bound for each sub-stratum and aggregate to determine the total lower bound for that cost center stratum. Take the total lower bound for each stratum divided by the total Medicare statistic of that stratum to calculate the error rate to be applied to that stratum (cost center). This error rate will determine the total extrapolated amount at the cost center level.

This same approach should be used to adjust the total claims expense/cost as well, if applicable.

Present the lower bound (or sample error) in the Draft and Final report. The work papers must include calculations of both the point estimate and lower bound estimate.

When Possible, Tabulate the Error Exactly

The Contractor should tabulate the amount of error exactly whenever possible. If an error has been identified, then the Contractor should determine whether it is systemic. If so, the total magnitude of the error on the cost report should be estimated and applied independent of sampling. For example, if the Medicare claims expense includes hearing aids, these payments should not be tested through sampling, but rather the procedure codes should be tabulated electronically and removed from the cost report in an adjustment separate from the sampling extrapolation. An exact tabulation removes uncertainty that may be attributed to statistical estimates.

Some areas where this may be applicable include non-allowed cost centers, non-covered services, provider contracts, recalculation using fee schedules, application of member benefits, determination of member cost share, and identification of duplicate payments.

This analytic approach is described in section 1.09 of the AICPA Audit Guide: Audit Sampling[footnoteRef:3], “Analytic Procedures – Procedures Applied to Every Item in a Population”. [3: American Institute of Certified Public Accountants, Inc. May 1, 2008. AICPA Audit Guide: Audit Sampling.]

The Contractor should note that in most cases, systematic errors should be identified and removed by the CAC prior to the claims database being provided to the MRC.

image1.jpeg