Attachment_3_Quality_Assurance_Reviews.pdf

QUALITY ASSURANCE REVIEWS

The Office of Management Information (OMI) of the Civil Division uses the services of an independent quality assurance (QA) reviewer to examine contractor work against the contract requirements and accuracy standards. The independent QA contractor checks the data entered by the Case Information Support Services contractor for accuracy, completeness, and adherence to the rules and guidelines found in the COR-approved manuals and any associated letters of instruction (LOI). Depending on the type of QA Review, the QA contractor uses source documents, manuals, LOIs, and observations to identify errors, inconsistencies, missing information, lack of documentation and correct data.

This document presents an overview and the accuracy standards for the following types of quality assurance reviews:

Mail Routing/Identification QA Review Case Classification QA Review Database Integrity Program QA Review

1. MAIL ROUTING/IDENTIFICATION QA REVIEW

QA reviews of Mail Routing/Identification will examine, in accordance with several criteria, the timeliness and accuracy of mail delivery of the almost 10,000 pieces of mail that come to the Civil Division each month. The Civil Division's Office of Management Information (OMI) requires a review of the sorting of one mail run per week, or as directed by the COR.

The Mail Routing/Identification QA review consists of six consecutive steps:

(1) Retrieving and copying a mail sample;

(2) Inspecting the sample to verify that it has been placed in the correct envelope for delivery;

(3) Inspecting the sample for timeliness by comparing the date received stamp with the released date stamp;

(4) Inspecting the sample for proper routing;

(5) Inspecting the sample for accurate identification; and

(6) Recording errors and observations.

1.1. Accuracy Rate Calculation

There are three accuracy rates that are automatically calculated by the Mail Routing/Identification report for each QA review: Mail Routing/Time Accuracy; Mail Identification Accuracy; and Combined Accuracy. The Combined Accuracy standard is 98%. A description of how the accuracy rates are calculated is provided below.

Attachment 3 Case Information Support Services

1.1.1. Mail Routing/Time Accuracy

The Mail Routing/Time Accuracy is calculated as follows:

(Total Routing and Time Fields – Total Routing and Time Field Errors) / Total Routing and Time fields x 100 = Mail Routing/Time Accuracy Rate percentage

1.1.2. Mail Identification Accuracy

The Mail Identification Accuracy is calculated as follows:

(Total Identification Fields – Total Identification Field Errors) / Total Identification fields x 100 = Mail Identification Accuracy Rate percentage

1.1.3. Combined Accuracy

The Combined Accuracy is calculated as follows:

(Total Routing and Time Fields – Total Routing and Time Field Errors) + (Total Identification Fields – Total Identification Field Errors) / (Total Routing and Time fields + Total Identification fields) x 100 = Combined Accuracy Rate percentage

2. CASE CLASSIFICATION QA REVIEW

There are three types of classification reviews performed by the QA Reviewer: Pre- Filing/Trial Classification, Appellate Classification, and Contractor QC Review Classification.

QA reviews of Case Classification will examine, in accordance with numerous criteria, the classification process; that is, of the creation of a DJ number and extraction and entry of data elements in an accurate, consistent and timely manner. OMI requires QA of 10 percent, of the cases classified each month.

The QA Review consists of four basic, consecutive steps:

(1) Retrieving and copying the sample items;

(2) Searching CIMS to determine that the document is a new case;

(3) Reviewing the sample court documents against the New Sheet and CIMS for accuracy; and

(4) Recording errors and observations.

2.1. Accuracy Rate Calculation

OMI has established three different accuracy categories: Caption, Content, and Combined. The Combined accuracy standard is 98%. All of these accuracy rates are calculated automatically. A description of how the accuracy rates are calculated is provided below.

2.1.1. Caption Accuracy

The Caption Accuracy is calculated using the following formula:

(Total Caption Words – Total Caption Word Errors) / Total Caption Words x 100 = Caption Accuracy rate percentage

2.1.2. Content Accuracy

The Content Accuracy is calculated using the following formula (note: the “Total

no. of Fields” and “Total Field Errors” figures do not include the caption field):

(Total No. of Fields – Total Field Errors) / Total No. of Fields x 100 = Content Accuracy rate percentage

2.1.3. Combined Accuracy

The Combined Accuracy is calculated using the following formula:

(Total Caption Words – Total Caption Word Errors) + (Total No. of Fields – Total Field Errors) / (Total Caption Words + Total No. of Fields) x 100 = Combined Accuracy rate percentage

3. DATABASE INTEGRITY PROGRAM QA REVIEW

QA reviews of the Database Integrity Program (DBI) will examine, in accordance with numerous criteria, the analysis of case listings accomplished by DBI personnel;

participation in the attorney interview process to assess its effectiveness in information gathering; the forms completed as a result of case updates obtained from the interview process; and accessing the case management system to ensure that all indicated updates have been correctly made in the automated database. As designated by the COR, each week QA reviews will be conducted on approximately five attorney case listings and/or DBI interviews.

The Quality Assurance contractor performs two types of DBI QA reviews: the File Review and the Interview Review.

3.1. File QA Review

In the DBI File Review, the QA Reviewer analyzes the Comprehensive Attorney Case List (CACL) (pulled from the DBI files and copied). The CACL is analyzed to determine that all updates to CIMS, as a result of information gathered at interviews by the DBI analyst, have been entered into CIMS and that all disposition forms completed during the interview are found in the files. To assist in the analysis of the CACL, the QA reviewer should use all of the tools available such as: the DBI Procedures Manual, the DBI QA Checklist, DBI LOIs, Lexis CourtLink, etc. The QA Reviewer does not attend the attorney interview in a DBI File Review. The determination of errors is mostly based on notes written on the CACL by the DBI analyst.

3.2. Interview QA Review

In the DBI Interview Review, the QA Reviewer accompanies the DBI analysts to attorney interviews to audit the information gathered. When accompanying the DBI analyst on attorney interviews, the QA Reviewer brings the CACL which the QA Reviewer has already analyzed for apparent anomalies. During the interview, the QA Reviewer takes detailed notes of any changes to the status of the attorney's cases that have occurred since the previous interview. Any issues or data elements that the QA Reviewer thinks were overlooked and should have been addressed in the interview are noted on the CACL and later identified on the QA review forms. The QA Reviewer does not take part in the discussion at any time and is merely present to observe.

3.3. Accuracy Rate Calculation

OMI has established three different accuracy categories: DBI Fields, Other Fields and Combined. The Combined accuracy standard is 98%. A description of how the accuracy rates are calculated is provided below.

3.3.1. DBI Fields Accuracy

DBI Fields Accuracy is calculated using the following formula:

(Total DBI Fields – Total DBI Field Errors) / Total DBI Fields x 100 = DBI Fields Accuracy Rate percentage

3.3.2. Other Fields Accuracy

The Other Fields Accuracy is calculated using the following:

(Total Other Fields – Total Other Fields Errors) / Total Other Fields x 100 = Other Fields Accuracy Rate percentage

3.3.3. Combined Accuracy

The Combined Accuracy is calculated using the following formula:

(Total fields – Total field Errors) / Total fields x 100 = Combined Accuracy rate percentage