Attachment_(8)_Case_Classification_and_Documentation_Procedures.doc

QA Review Procedures Document Change History Record

Case Management SYstem Quality Assurance Review Procedures

U.S. Department of Justice

Civil Division, Office of Management Programs

Office of Management Information

February 9, 2016

DOCUMENT CHANGE HISTORY RECORD

0.0

Table of Contents

61.0

INTRODUCTION

72.0

CLASSIFICATION QA REVIEWS

72.1 Classification QA Reviews

72.1.1 Pre-Filing/Trial Classification Reviews

72.1.2 Appellate Classification Reviews

82.1.3 Contractor QC Reviews

82.2 Contractor Processing

82.3 Sample Selection

92.4 QA Review Procedures

102.5 Completing the QA Review

102.5.1 Access the Classification QA Review

102.5.2 Complete a Classification Review

132.6 Accuracy Rate Calculation

132.6.1 Caption Keying Accuracy

142.6.2 Content Accuracy

142.6.3 Combined Accuracy

153.0 DATABASE INTEGRITY (DBI) QA REVIEW

153.1 DBI QA Reviews

153.1.1 DBI File Review

153.1.2 DBI Interview Review

163.2 Contractor Processing

163.2.1 Reviewing Pre-Filing/Trial Stage Cases

173.2.2 Reviewing Appellate Stage Cases

193.2.3 Parties

193.2.4 Relief

203.2.5 Government Exposure

203.2.6 Case Time

213.2.7 Special Codes

223.2.8 Case Disposition and Closure

223.2.9 Notes Screen

223.2.10 Monitored Cases

233.3 Sample Selection

233.3.1 Access the Database Integrity Module

243.5.2 To Select a Scheduled Interview for Review

243.4 QA Review Procedures

253.5 Completing the QA Review

253.5.1 Access the DBI QA Review

253.5.2 To Complete a DBI Review

273.6 Accuracy Rate Calculation

283.6.1 DBI Fields Accuracy

283.6.2 Other Fields Accuracy

283.6.3 Combined Accuracy

294.0 MAIL ROUTING/IDENTIFICATION QA REVIEW

294.1 Mail Routing/Identification QA Reviews

294.2 Contractor Processing

304.3 Sample Selection

304.4 QA Review Procedures

314.5 Completing the QA Review

324.6 Accuracy Rate Calculation

324.6.1 Mail Routing/Time Accuracy

334.6.2 Mail Identification Accuracy

334.6.3 Combined Accuracy

345.0 ATTORNEY TIME REPORTING QA REVIEWS

345.0.1 Participation of the Legal Staff

345.0.2 Receipt and Entry of Time Reports

345.0.3 Reference Terms

355.1 Attorney Time Reporting QA Reviews

355.2 Contractor Processing

365.2.1 Reviewing Multiple Days of Reported Time

365.2.2 Keeping Track of the Calendar Date

365.2.3 Time Reported During Leave

365.3 Sample Selection

375.4 QA Review Procedures

375.4.1 To Access the Attorney Time QA Review

375.4.2 To Complete an Attorney Time QA Review

395.5 Accuracy Rate Calculation

406.0 QA REVIEW RESULTS SUBMISSIONS

406.1 Classification /DBI/Mail Letter of Instruction

406.2 Correction Verification

417.0 REPORTING PROCEDURES

417.1 Daily Reports

417.2 Monthly Reports

417.4 Fiscal Year Report

42Appendix A - Pre-Filing/Trial Classification Field Evaluation Instructions

5Appendix B - Appellate Classification Field Evaluation Instructions

5Appendix C – DBI Evaluation Instructions

7Appendix D – Mail Routing/Identification Evaluation Instructions

3Appendix E – Attorney Time Reporting Evaluation Instructions

1.0 INTRODUCTION

The QA contractor provides quality assurance (QA) services to the Civil Division of the Department of Justice (DOJ). These procedures are in response to delivery orders issued by the Office of Management Information (OMI) for QA of the Case Information Support services provided to the OMI by another government contractor.

The Civil Division represents Federal agencies in litigation. All cases handled by the Civil Division are tracked on an automated database called CASES. The Case Management System database (CASES) contains information on active and closed Civil Division cases. It tracks detailed information on case status from receipt through the trial and appellate stages. The Civil Division will make numerous entries over the life of a typical case and will use this information, not only in tracking civil litigation information, but also in its planning and budgeting process, in determining attorney case load, among other purposes.

The Case Information Support Services contractor is responsible for:

· Identifying and entering new civil cases involving the Government;

· Entering the mandatory and discretionary data elements for the new cases;

· Updating the case records as litigating activities occur;

· Entering case disposition/closure information;

· Identifying and routing incoming mail;

· Entry of time records; and

· Quality Control.

To ensure that the contractor services meet contractual quality requirements, the Civil Division instituted a QA program that requires an independent, third party to review contract deliverables and processes on a systematic basis. OMI’s QA program evaluates the accuracy of the case information support services contractor's Classification, Database Integrity (DBI), Mail Routing, Attorney Timekeeping, and Contractor quality control activities. This manual documents the Civil Division procedures for conducting quality assurance assessments for OMI.

2.0 CLASSIFICATION QA REVIEWS

When a civil lawsuit involving the federal government is filed in court, the Attorney General and/or appropriate U.S. Attorney are served with documents called a "Summons” and “Complaint." These documents are forwarded to the Civil Division for handling. Within one working day of receipt of these documents, the OMI Mail Unit will send them, along with any other documents used to create a new case, to the Classification Unit. The Classification Unit will then extract pertinent information from the court documents to create a new record in CASES. This new record will become the foundation for tracking the case throughout the entire litigation process. The information contained in CASES is used for tracking case progress, resource management and budgeting. Therefore, it is critically important that the information in CASES be complete and accurate.

When a new case is received, the classifier assigns a unique identification number known as a Department of Justice (DJ) Number. Numerous data elements are extracted from the court documents. Court documents, such as the summons and complaint, are used to determine the DJ Number and other relevant data.

2.1 Classification QA Reviews

There are three types of classification reviews performed by the QA Reviewer: the Pre-Filing/Trial Classification Review, the Appellate Classification Review and the Contractor QC Review.

2.1.1 Pre-Filing/Trial Classification Reviews

The Pre-Filing/Trial Classification Review is a review of a pre-filing/trial stage case. A DJ Number can be created prior to a case being filed in court; this is known as a pre-filing stage. When a case is filed in court, it is referred to as the trial stage. The Pre-Filing/Trial Classification Review is a review of a sample consisting of cases in the pre-filing and trial stages.

2.1.2 Appellate Classification Reviews

The Appellate Classification Review is a review of an appellate stage case. The appellate stage occurs when a case is filed in an appellate court. The appellate stage has numerous unique fields that are not applicable to the pre-filing/trial stage; therefore, the two stages are reviewed separately.

2.1.3 Contractor QC Reviews

The Contractor QC Review is similar to the Pre-Filing/Trial and Appellate Classification QA Reviews in the regard that the QA Reviewer ensures that there are no discrepancies between CASES and the court documents. However, instead of reviewing the classifier, the QA Reviewer ensures that the Contractor QC Reviewer has accurately reviewed the sample.

2.2 Contractor Processing

Mail for the Civil Division is received by the Mail Identification Unit located in the OMI office. The Mail Identification Unit has until the end of the business day to identify and process the mail. AG Served mail and documents not associated with an existing case in CASES are forwarded to the Classification Unit.

The Classification Unit will then confirm that the unmatched material sent by the Mail Identification Unit is not associated with a case. If the document is not associated with a case, the Classification Unit is responsible for assigning the DJ Number and creating the original record in CASES. This new record becomes the foundation on which the entire case record is built. The Classification Unit must accurately and efficiently classify the court documents. Delays or inaccurate classifications can impede the litigation process.

After the Classification Unit enters the initial information on a new case, two cards are printed which display the classification entries. These cases are referred to as the “New.”

The Classification Unit performs quality control after cases are entered into CASES. Cases are then returned to the Mail Identification Unit for branch dissemination. New cases are picked up by the Mail Identification Unit six times during the day: 8:30 a.m., 10:30 a.m., 1:30 p.m., 2:45 p.m. (OIL Appellate cases only), 3:30 p.m. and 5:15 p.m.

2.3 Sample Selection

The QA Reviewer selects a sample of the new classified cases prior to the quality control performed by the contractor, except for Contractor QC Reviews which are selected after the QC review. The new Pre-Filing/Trial and appellate stage cases are located in an outgoing box in room 8006. The QA Reviewer should also select sample cases from the “Specials” box located in room 8011. “Specials” are documents that require special attention and are separated from the other new cases. The QA Reviewer retrieves cases for review prior to the six designated mail pickup times and returns the cases as soon as possible. A sample is taken that encompasses all of the different classifiers, pre-filing/trial and appellate stage cases, as well as, DJ Number prefixes. The sample documents are then copied being sure to copy the OMI Received date stamp on the back of the Blocking Tag or the first page of the documents as well as all of the relevant information on the AG Service envelope. Care should be taken to not damage the documents since these are often the original documents for the case. The originals that were taken from room 8006 are then given to the Mail Identification Unit so that the mail processing is not interrupted. “Specials” should be returned to the box in 8011 when copying is finished paying attention to the Mail ID Unit pickup times. These cases have a shorter turnaround time and need to be sent back to the branch within 24 hours of receipt.

Documents that have been Contractor QC’ed are found in two places. The first location is in the outgoing boxes in room 8006 mixed in with other non QC’ed cases. These will have yellow contractor QC form attached. The second location is in room 8011 where the “Specials” are located. These will have a green contractor QC form attached. Both types of Contractor QC’ed cases are pulled for the sample. The yellow form QC cases can be returned to the Mail Identification Unit when copying is finished. “Specials” (green form QCs) should be copied and returned immediately to the box in room 8011 paying attention to the Mail ID Unit pickup times. These cases have a shorter turnaround time and need to be sent back to the branch within 24 hours of receipt.

2.4 QA Review Procedures

The QA Review consists of four basic consecutive steps:

1. Determining and collecting the sample items;

2. Searching the CASES database to determine if the new case is a duplicate;

3. Reviewing the sample court documents against the new case data card and CASES for accuracy; and

4. Recording errors and observations in the OMI Tasks Tracker application.

The Pre-Filing/Trial Classification Review and Appellate Classification Review are completed for each individual classifier. The Pre-Filing/Trial Classification Review contains a sample of ten cases from each classifier. The Appellate Classification Review contains a sample of two cases for each classifier.

The Contractor QC Review is completed for each QC Reviewer. Each review contains a sample of ten cases. Documents that have been Contractor QC’ed have a yellow form for regular NEWs or a green form for Specials, titled “ISSI Case Classification Quality Control Review”, attached to them. This form will provide the name of the classifier as well as the name of the QC Reviewer.

The QA Reviewer reviews the fields on the new case data card and data in CASES against the copied court documents following the Pre-Filing/Trial Classification Field Evaluation Instructions (Appendix A) or the Appellate Field Evaluation Instructions (Appendix B) depending on the type of review. The QA Reviewer should also refer to the tools used by the classifiers: the classification chart, classification procedures manual, classification Letters of Instruction (LOIs) and the abbreviation/acronym list when completing a review.

As each case is reviewed, the QA Reviewer enters pertinent information and errors into the OMI Tasks Tracker which calculates the accuracy rates for the entire sample and the results for each case reviewed.

2.5 Completing the QA Review

The OMI Tasks Tracker is used to capture the results of the QA Review.

2.5.1 Access the Classification QA Review

To access the OMI Tasks Tracker, follow the instructions below:

1. Click Start > All Programs > Internet Explorer. The Internet Explorer window displays.

2. In the address bar, type http://omi. The OMI intranet window displays.

3. Select OMI Tasks Tracker from the menu located on the left side of the window.

4. Select QA below the OMI Tasks Tracker header.

2.5.2 Complete a Classification Review

1. Select the type of review being performed:

· To complete a Pre-Filing/Trial Classification Review, select Case Review from the QA Menu.

· In the Classifier field, select the classifier listed on the new case data card from the drop-down list.

· To complete an Appellate Classification Review, select Appellate Review from the QA Menu.

· In the Classifier field, select the classifier listed on the new case data card from the drop-down list.

· To complete a Quality Control Classification Review, select QC Review from the QA Menu.

· In the QCer field, select the QC reviewer listed on the “Case Classification Quality Control Review” form from the drop-down list.

2. Select the Date of Review using the calendar or enter a date. The Date of Review is the first business day of the month being reviewed.

3. In the column under the DJ Number field, enter the DJ Number listed on the new case data card. If the case had been deleted from CASES enter “D-“before the DJ Number.

4. The system will automatically count the number of words in the case caption. If it differs from the number of words that are printed on the case card enter the correct number in the column under Cap Wrds.

5. Count the number of fields reviewed (with the exception of the Caption field) and enter the total number in the column under # of Fields. This number varies from case to case depending on the number of plaintiffs, defendants, agencies and/or other ID numbers involved in the case. Even when information is not included in a field, it is still counted as part of the total “No. of Fields” figure since the field is still reviewed for possible missed information. The AG served date has three additional related fields that are not present on the new case data card but are entered in CASES by the classifiers; and therefore, the AG served date field should always be counted as four fields. The four fields are: AG served date, What was AG Served, How was AG Served and Mail Registered/Certified Number. The US Attorney’s Office Served Date field should always be counted as two fields since the US Attorney’s office number field (USAO Number) is also quality assured but is not present on the new case data card. The relief sought field is counted as two fields: Opposition Sought and Government Sought.

6. All of the relevant fields on the new case data card and in CASES are then reviewed for accuracy. To complete the review, the QA Reviewer utilizes all of the tools used by the classifiers. Certain fields can have multiple entries. For example, the party field can have many parties listed for one case. When there are multiple parties, agencies or other ID numbers involved in a case, they may not all be listed on the new case data card. Another example involves the Branch/Section fields. Some Civil Division Branches are broken into sections. It is important to check the Section field in CASES which is not on the Case Card to make sure the classifier entered this information correctly. The QA Reviewer will need to inspect CASES to verify that these data elements were entered in the system accurately. To access these data elements in CASES, open the CASES database and a search filter window will appear. Double click on Caption in the List of Columns field (Note: Other search criteria, i.e. Docket Number, can be used to retrieve the QA’d case. However, the search method used should be able to detect possible duplicates). Change the Operator field to Contains and enter a portion of the caption in the Expression field and then click the green check mark. A list of all cases in the database that have captions with the searched information will appear. Make sure there are no potential duplicate cases. Double click on the case being QA’d and this will bring up the case detail screen. This screen provides general information about the case such as the DJ Number, parties, caption, sought amounts, primary agency, etc. Click on the appropriate stage for the case being QA’d under the Stages Tab to access additional information (i.e. AG Service, USAO Service, branch/section, judicial district, etc.).

7. The QA Reviewer reviews the case caption for spelling, abbreviations, missing or incorrect information and extraneous words in the case caption. Note: The Contractor QC review has one additional field not listed in Appendix A or B, the Short Name (SN) field. When looking at contractor QC review cases, this field should be checked to make sure it matches the Caption field up to the maximum characters for the field. If it does not match the Caption field it is a Short Name field error.

8. The QA Reviewer searches CASES to ensure that the case in the review does not already exist in the system. If more than one case is returned, the other cases are reviewed to determine if the case was already entered in CASES. If so, this is a DJ Number error that should be reported in the QA review. If it is determined that a duplicate case exists in CASES but was entered on the system after the case that is being reviewed, this is not an error for the classifier who originally entered the case. When this situation occurs, the QA reviewer should bring this to the attention of OMI so that the duplicate case can be removed from CASES.

9. When an error is identified, the documentation for the case should be designated as an exhibit. The error should be noted on the new case data card and the section of the attached documentation for the case that confirmed the error should be highlighted.

10. If no errors are identified in the case, click the Save button.

11. If multiple errors are found for a single DJ Number, it is important to create a separate record for each error. For example, two parties listed in CASES that are incorrect, each error would be entered as a separate record. Click Add DJ Number to add additional errors and enter the DJ number again. The Branch, Cap Wrds and # of Fields columns do not need to be populated again.

12. Enter “1” in the column pertaining to the error type.

13. In the Error field, enter the corresponding exhibit number in parentheses and a description of the error.

14. In the Correction field, enter the corresponding exhibit number in parentheses and a description of the correction, click the Save button.

15. To add another record, click Add DJ Number.

16. To complete the QA Review, the QA Reviewer writes a summary report of the results and any recommendations from the review in the Analysis of Results and Recommendations field.

2.6 Accuracy Rate Calculation

OMI has established three different accuracy categories: Caption Keying, Content, and Combined. All three accuracy rates are calculated automatically by the OMI Tasks Tracker. A description of how the accuracy rates are calculated is provided below.

2.6.1 Caption Keying Accuracy

The Caption Keying Accuracy is calculated using the following formula:

(Total Caption Words – Total Caption Word Errors) / Total Caption Words x 100 = Caption Keying Accuracy rate percentage

2.6.2 Content Accuracy

The Content Accuracy is calculated using the following formula (note: the “Total no. of Fields” and “Total Field Errors” figures do not include the caption field):

(Total No. of Fields – Total Field Errors) / Total No. of Fields x 100 = Content Accuracy rate percentage

2.6.3 Combined Accuracy

The Combined Accuracy is calculated using the following formula:

((Total Caption Words – Total Caption Word Errors) + (Total No. of Fields – Total Field Errors)) / (Total Caption Words + Total No. of Fields) x 100 = Combined Accuracy rate percentage

3.0 DATABASE INTEGRITY (DBI) QA REVIEW

The DBI program was designed to supplement the Division’s efforts to ensure that all data elements are correctly entered, and that the database is current, consistent and accurate. As the lawsuits progress, the DBI unit ensures that CASES is updated with the current information for the case. This is accomplished by reviewing the Comprehensive Attorney Case Listing (CACL) for each attorney and conducting “attorney interviews” in which the attorney is asked specific questions about the case and its progress. The CACL is the primary tool used by the DBI analysts to complete DBI reviews. The CACL is a printout derived from CASES containing information on all open cases assigned to an attorney.

Each DBI analyst is assigned a group of Civil Division attorneys and meets with each individual attorney approximately once every three months (bi-annually for immigration attorneys). The QA Reviewer attends and/or reviews various interviews to assure that the right questions are asked and that the information obtained at an interview is updated accurately in CASES.

3.1 DBI QA Reviews

There are two types of DBI QA Reviews: the DBI File Review and the DBI Interview Review.

3.1.1 DBI File Review

In a DBI File Review, the QA Reviewer retrieves the CACL and related disposition forms from the DBI files, located in either room 8017 or 8023 (the room will vary depending on the year), and makes a copy. The copy is analyzed to determine that all necessary updates to CASES, as a result of information gathered during the interview, have been entered and that all disposition forms completed during the interview are entered appropriately. To assist in the analysis of the CACL, the QA Reviewer should use all of the tools available such as: the DBI Procedures Manual, the DBI Evaluation Instructions (Appendix C), DBI LOIs, Lexis CourtLink, etc. The QA Reviewer does not attend the attorney interview in a DBI file review. The determination of errors is based on notes written on the CACL by the DBI analyst, any data anomalies and missing information identified and a thorough review of CASES by the QA Reviewer.

3.1.2 DBI Interview Review

For the DBI Interview Review, the QA Reviewer accompanies the DBI analyst on an attorney interview to audit the information gathered and the pertinent questions asked. Prior to attending an attorney interview, the QA Reviewer analyzes the CACL for apparent anomalies and missing information. When the QA Reviewer accompanies the DBI analyst to the attorney interview, the QA Reviewer brings the analyzed CACL. During the interview, the QA Reviewer takes detailed notes of any changes to the status of the attorney's cases that have occurred since the previous interview. Any issues or data elements that the QA Reviewer thinks were overlooked and should have been addressed in the interview are noted on the CACL and later identified on the QA review forms. The QA Reviewer does not take part in the discussion at any time and is merely present to observe.

To access the CACL, open the DBI section of the OMI Tasks Tracker. See Section 3.3.1. Next select “Reports” from the file tree on the left side of the screen and then click on the “CACL Report” option. Select the attorney being interviewed by accessing the attorney dropdown menu on the top of the screen and searching for the attorney’s name. Click on the attorney’s name to produce the CACL report for the selected attorney. To print the CACL, click on the print icon to the right of the report.

3.2 Contractor Processing

After printing the CACL, click on the “C” icon in the navigation bar at the top of the CASES screen to return to the Case Management/Case Query display, press F8, enter the DJ Number and then press enter. This will bring up the case detail screen. This first screen provides general information about the case such as the DJ Number, caption, amount of money sought by the government and/or the opposition, amount of money awarded to date, government exposure, current primary attorney and reviewer, and status of the case (whether it is open or closed). The QA Reviewer analyzes all of this information during the review process using all of the tools available such as: the DBI Procedures Manual, the Classification Procedures Manual, the DBI Evaluation Instructions (Appendix C), DBI LOIs, Lexis CourtLink, Abbreviation and Acronym lists, disposition forms, etc.

3.2.1 Reviewing Pre-Filing/Trial Stage Cases

The QA Reviewer first verifies the appropriate classification abbreviations and acronyms have been used in the caption. The caption is originally entered in the system by the Classification Unit and should reflect the original court caption for the case as closely as possible. The Short Name field should also be checked to make sure it matches the caption (at least up to the maximum characters for the field). If it does not match, the DBI Analyst should ask the attorney if the variant Short Name field is accurate. If the attorney indicates that it is accurate, the DBI analyst should enter special code 520 (Variant Short Name Verified as Accurate) in the Special Code field. If the attorney indicates that it is not accurate, the DBI analyst should update the field to match the caption. The QA Reviewer then ensures that the primary attorney listed in CASES matches the attorney name on the CACL being reviewed. If the name does not match, the DBI analyst should ask the attorney at the interview whether he or she is the lead attorney on the case. If the case has been verified as filed, there will be a "Yes" in the "Has this case been filed?" field. The QA Reviewer verifies that the stage has been updated to filed, if the case is no longer a pre-filing. The QA Reviewer verifies that the "Filing Date" field has a date input into this field.

Next, the QA Reviewer verifies that the "Government Role" field corresponds to the case caption. For instance, if the government is taking a defensive stance in the case the "Government Role" will be marked as "D" and the government agency or agent for the government will appear on the right of the "v" (the abbreviation for versus) in the case caption.

Exception: The Government Role in bankruptcy cases is usually affirmative even though they are listed in the caption as a defendant.

The QA Reviewer then verifies that the "DJ Number" and "Branch" correctly correspond to one another and to the numbering system established by the DOJ Classification Chart. For instance, in Federal Programs Branch cases, the DJ Number often begins with the number "145" or “35.”

3.2.2 Reviewing Appellate Stage Cases

After disposition of a trial, the parties involved have 30 to 120 days, depending on the court hearing the case, to file a brief appealing the court's decision. The party filing the appeal in an appellate case is the "Appellant." The party defending the court's previous decision is the "Appellee." To begin the review of an appellate stage case, the QA Reviewer enters the DJ Number at the Case Selection screen in CASES.

In the Case Detail screen, the QA Reviewer first verifies the appropriate classification abbreviations and acronyms have been used in the caption and that the Short Name field matches the Caption field (see Section 3.2.1). Next, the QA Reviewer examines the case stage information to ensure that the information is complete and current with the litigation progression of the case. For a case at the appellate stage, there may be several stages entered in CASES.

An interlocutory order or judgment is a provisional or temporary decree that does not determine the issues at trial but directs some further proceeding preliminary to the final judgment. An interlocutory appeal is an appeal of issues that are tangent but pivotal to the case. If the reason for the appeal is listed as "interlocutory," the lower level stage will still be undisposed.

An appellate memorandum (appeal memo) is a memo drafted by the DOJ attorneys seeking permission of the Solicitor General to request that the court of appeals review a trial court decision. A certiorari memorandum (cert memo) is a memo drafted by the DOJ attorneys seeking permission of the Solicitor General to either request the Court of Appeals rehear a case or to petition the Supreme Court. The determination as to whether or not the government should pursue the appeal will come from the Office of the Solicitor General within 30 to 60 days of the date when the petition for appeal was originally filed. Each of these memorandums is given its own stage in CASES and is disposed appropriately.

The QA Reviewer checks the pre-filing/trial stages of the case to verify that all of the relevant case information is in CASES. The QA Reviewer examines the dates that the case was received and disposed from different stages of litigation to get a perspective on the time span for which the case has been opened. If there is any information missing from the pre-filing/trial stage (especially if there is a disposition missing once the case has progressed to an appellate stage), the QA Reviewer notes on the CACL that this is an issue that should be raised in the attorney interview by the DBI analyst.

In the appellate memo stage, the QA Reviewer verifies the branch, the attorney, and the reviewer of the case to see that the information entered is appropriate to the case. The handling for most appellate stages is "P" for personally handled. The QA Reviewer ensures that there is a reason for appeal ("Reason"). If there is not a “Reason” then the QA Reviewer should annotate this on the CACL as an issue that the DBI analyst should address during the attorney interview. A case may be appealed because of the issues involved in the case and/or because of the attorney fees involved in the case.

The QA Reviewer checks the government’s participation in the appeal, “Govt. Part”, to see that it is congruent to the caption.

In the Appellate Court stage, the QA Reviewer verifies that the attorney and reviewer entered for that stage are correctly entered. The QA Reviewer also checks the handling code for the case, "Handling", the reason for the appeal, "Reason", and the government role in the appeal to verify that they correspond to what has been entered in any lower level stage such as an appellate memo or trial stage.

Instead of a Civil Action Number appellate stage cases have a docket number, "Docket No.". If the docket number is missing, the QA Reviewer notes this as something the DBI analyst should ask about in the interview.

If the case has proceeded to a second stage of appeal, the QA Reviewer examines any additional appellate stages to ensure that all of the necessary case information has been entered at the appropriate stages of litigation. If any data elements are missing from previous appellate stages of the case, the QA Reviewer annotates on the CACL that the DBI analyst should mention it in the interview.

3.2.3 Parties

When the QA Reviewer clicks on the "Parties" tab the screen displays the parties involved in a case. The first field listed is the "Name" field. The second field is the "Role" field which shows the role of each party in a case. The five roles that may appear are: Plaintiff “P,” Defendant “D,” Amicus “A,” Intervenor “I,” and Unknown Defendants “U.” The third field titled "Role Desc," displays the description of the role of the party. For example, "D" is Defendant. The fourth field, "Start Date" is the date the party became active in the case (Note: this is different than the Start Date field on the Stage Detail page).

The fifth field, "End Date," is the date that the party became inactive within the scope of the case. The last field is "Secured Party." All cases in the pre-filing stage will have a "Yes" in this field, and most viewers of CASES will not have access to view the parties involved in this litigation. If the case has proceeded to trial, then a filing date is requested and entered into CASES. The “Secured Party” field is changed to "No," meaning that the Secured parties will be visible to all users on CASES. The QA Reviewer checks the parties for proper use of abbreviations and acronyms, correctness of the role and missing or duplicate parties.

3.2.4 Relief

The relief involved in a case can be seen on the case detail screen in CASES. The relief section displays the kind of relief that has been sought by and awarded to the government and/or opposing parties. There are four segments to the relief synopsis: the "Govt - $ Sought," the "Oppn - $ Sought," "Govt - Awarded to Date," and the "Oppn - Awarded to Date."

Monetary relief sought by the opposition is displayed in CASES to the right of the words "Oppn - $ Sought". There are four possible entries for relief sought: "U," "Y," "E," or "N”. The "E,” indicates that only equitable relief is being sought in the case. A “U” is used when the amount of money that is being sought in a case is unspecified at the moment. If the amount of monetary relief sought by the opposition is unspecified, the QA Reviewer notes on the CACL that the amount being sought by the opposition has been left unspecified and that the DBI analyst should ask the attorney in the interview whether or not the opposition is seeking a specific amount. A "Y" signifies that there is a specific amount being sought, and a dollar amount should be entered next to "Amt." An "N" signifies that there is no monetary relief being sought.

The "Govt - $ Sought" field is set up in a similar fashion to the "Oppn - $ Sought" field, except that all the information pertains to the relief sought by the government. The QA Reviewer verifies the relief information corresponds with the government role for the particular case.

Attorneys working on affirmative Commercial cases who have recorded attorney time on discovery or settlement related activities should be asked in interviews if the government is seeking a specific amount. Specific amounts should also be requested from attorneys working on defensive Torts cases who have recorded attorney time on defensive discovery and settlement activities. If relief has been awarded, the QA Reviewer examines the relief detail information in CASES and confirms that all of the information entered is correct and appropriate for the case.

The QA Reviewer should examine the relief for appellate cases to verify that it is accurate with regard to the other information (i.e. government role) in CASES.

3.2.5 Government Exposure

The "Government Exposure" field in CASES records the amount of risk that the government believes the case represents. If the opposition is seeking monetary relief, greater than five million dollars, the DBI analyst should ask the attorney the amount of government risk. This amount should be entered into CASES and is reviewed by the QA Reviewer.

3.2.6 Case Time

CASES maintains a record of all the time reported by the government attorneys, reviewers and paralegals working on each case. The time reporting function can be accessed by clicking on the clock icon in the navigation bar at the top of the CASES screen. All of the time on the system is entered according to DOJ assigned activity codes. Each activity code is to be used in a specific situation in the progression of a case.

The QA Reviewer examines the time activities entered for each case on the CACL being reviewed. If there are time activities entered in CASES that do not correspond to the stage of the case or conflict with other information in the system, the QA Reviewer looks for a note entered in CASES explaining the use of the conflicting activity code. For example, if an attorney is reporting “Court of Appeals” time to a case that does not have an appellate stage, there should either be a note explaining the anomaly or the DBI analyst should ask the attorney at the interview if this time entry is correct. One discrepancy with this rule is Administrative Claims. Administrative Claims are always in the pre-filing stage, but the time is recorded in the trail stage codes.

The QA Reviewer reviews all the screens for time entered in CASES to ensure correct time codes are recorded to the case. The “last time reported” field in CASES and on the CACL shows the last time that was reported to the case. The QA Reviewer compares this last time reported date to the received date for the case to assess the amount of activity occurring in the case. If a case has not had time reported for six months, the DBI analyst should specifically ask the attorney why the case has been inactive. Any information provided by the attorney should then be included in a note in CASES.

Reviewing the time reported and activity codes for each case is an important step in identifying changes in the status of a case and to identify incorrect information in CASES. For example, if the attorney has started to report pleadings and motions time for a pre-filing stage case, this may indicate that either the case has been filed, and the analyst should ask the attorney for the filing date, civil action number, and relief sought amount for the case or that there is a potential attorney time reporting error.

3.2.7 Special Codes

The DBI analysts are responsible for updating the special codes field in the CASES database for the 500 series. Special code information for each case can be seen by clicking on the "special code" tab on the case detail screen in CASES. All special codes are verified by the QA Reviewer during the quality assurance review. Special codes relating to monetary relief include 504, 505, 515, and 516. Special codes 504 (affirmative) and 505 (defensive) are the codes used to indicate relief sought greater than or equal to $5 million is accurate and has been verified by the attorney. Special code 515 is entered into CASES if the amount sought is exaggerated, but the attorney has not provided a government risk amount. A note must be entered stating the name of the attorney and the date the attorney did not provide the risk information. Special code 516 is entered into CASES if the amount sought is exaggerated, and the attorney has provided an estimate of the government risk. The estimated risk amount is entered into the "Government Exposure" field in CASES. A note is entered stating the name of the attorney, providing the risk information, the date the information was obtained, and the risk amount provided. Note: this procedure is not necessary for monitored cases.

Some special codes have expiration dates. When the expiration date expires and the situation warranting the special code is still relevant, the DBI analyst should ask the attorney about the information pertaining to the special code. If the special code is still applicable, it should be re-entered into CASES to reset the expiration date. For example, special code 517 is used when all of the stages for a case are disposed and the attorney wishes to keep the case open. If special code 517 was entered for a case and the expiration date expired prior to the interview, the DBI analyst should ask the attorney if the case should still remain open. If so, the code should be re-entered into CASES to reset the expiration date. For a list of all of the relevant special codes see the contractor’s DBI Procedures Manual.

3.2.8 Case Disposition and Closure

One of the primary duties of the DBI analyst is to confirm that CASES reflects the actual disposition of the cases. At the interview, as each case is discussed, the attorney notifies the DBI analyst if a case has been disposed, if it is still pending, or if it can be closed. If the case has been disposed, the DBI analyst will complete a disposition form and have the attorney sign it. The DBI analyst must additionally obtain the signature of the reviewer for fraud cases, national court cases, or cases involving monetary relief. The type of disposition, the date disposed, the closure date, and, if applicable, relief information are completed on the disposition form. The QA Reviewer notes these dates and disposition types and writes them on the CACL during the interview. The most commonly used disposition types for a case at the pre-filing and trial stages are listed on the back of the “Pre-Filing/Trial Stage Disposition and Relief Report” disposition form. If the attorney indicates that the disposition closes the case, the DBI analyst should enter the closure into CASES. However, a case should not be closed until the appeals period has expired. Depending on the court hearing the case, an appeals period ranges from 30 to 120 days. If the case cannot be closed due the appeals period a note should be entered into CASES stating that the case can be closed after the appeals period has expired.

3.2.9 Notes Screen

The QA Reviewer clicks on the "Notes Screen" tab of the case detail to view the notes entered for a case. Notes are entered into CASES to explain anomalies and gaps in cases that are due to special circumstances and not due to contractor error. The QA Reviewer should check each note entered to make sure that it is relevant to the case and for any spelling or content errors. Any note screens containing anomalies are printed and included in the review as examples. For example, if a note in CASES states that the DBI analyst is awaiting a signed disposition from the reviewer, and the disposition has been entered, the note is no longer relevant and can be deleted.

3.2.10 Monitored Cases

Monitored cases are cases in which the Civil Division has given the primary responsibility to pursue litigation either to the United States Attorneys’ Office or to the agency involved in the case. However, Civil Division attorneys reserve the right to monitor the case in an advisory capacity and to be apprised of the litigative progress of the case. Civil Division attorneys often have very little or no information on the cases they are assigned to monitor. For monitored cases, the Civil Division attorney may advise the U.S. Attorney's Office to provide status on the progress of the case. Ideally, the information in CASES about the Division's monitored cases would be updated as regularly and dependably as the information on personally and jointly handled cases.

Typically, the DBI analyst should ask the attorney about the status of each monitored case on the CACL. If there are numerous monitored cases on an attorney’s CACL, it is sufficient for the DBI analyst to ask the blanket statement: “Do you know of any changes to your monitored cases?” If the attorney provides information on a monitored case and the DBI analyst fails to update CASES or enters the information incorrectly, this is marked as an error on the QA Review. The QA Reviewer notes on the CACL those monitored cases for which there has been no recorded activity in a significant length of time as cases to be asked about in the interview.

3.3 Sample Selection

Sampling for both the DBI File Review and the DBI Interview Review is conducted by using the OMI Tasks Tracker. The DBI analyst is given five working days after the attorney interview to allow for the entry of all updates gathered in the interview. Any attorney interview chosen for a DBI QA Review will have to be completed no later than eight working days prior to the end of the month. This will allow adequate time for the DBI analyst to make updates and the QA Reviewer to conduct the QA Review.

3.3.1 Access the Database Integrity Module

To access the OMI Tasks Tracker, follow the instructions below:

1. Click Start > All Programs > Internet Explorer. The Internet Explorer window displays.

2. In the address bar, type http://omi. The OMI intranet window displays.

3. Select OMI Tasks Tracker from the menu located on the left side of the window.

4. Select DBI below the OMI Tasks Tracker header.

3.5.2 To Select a Scheduled Interview for Review

1. When the Database Integrity Module opens, Tasks Tracker will display the Scheduled Interviews page which lists all interviews scheduled by the DBI analysts from the current date. This listing will include both confirmed and unconfirmed interviews. Unconfirmed interviews are marked with an asterisk next to the attorney’s name.

2. Click on the Assigned Staff column heading in the upper right corner of the page. This will cause the page to be alphabetically sorted by the DBI analyst’s name.

3. Randomly select a name from the Interviewee column that corresponds to the DBI analyst that is being reviewed. Be sure that the interview has been confirmed by checking the Interviewee column, the interview is In-Person by checking the Form column, the interview is cyclical by checking the Type column and the date allows enough time for the analyst to complete updating the record by checking the Date column.

4. Click on the selected attorney’s name. This will open the Interview Form listing the details associated with this interview. Click on the QA Interview dropdown box and select Yes. Print the page by clicking the Print Button or note the Attorney’s name and branch/section. This will be used to retrieve the information for the review.

5. For a DBI File Review retrieve and copy the CACL and associated forms from room 8017 or 8023, See Section 3.1.1. For a DBI Interview Review print the CACL from the OMI Tasks Tracker to begin the pre-interview preparation.

3.4 QA Review Procedures

The DBI QA Review is performed no sooner than five working days after the interview to allow for the entry of all updates gathered during the interview. Like the DBI File Review, the QA Reviewer retrieves a copy of the filed CACL and disposition forms. The QA Reviewer then confirms that all of the updates were entered into CASES by the DBI analyst. The QA Reviewer also verifies that any anomalies overlooked by the DBI analyst during the interview are still present in CASES. If an error or anomaly is found, the QA Reviewer prints all the screens containing errors for inclusion with the QA Review Forms and highlights the data elements that are in error on the printouts. Additionally, the QA Reviewer writes directly on the printout explanations of the errors and/or anomalies and the necessary corrections. All the printouts for one review are grouped in a logical, sequential order and marked alphabetically as exhibits.

3.5 Completing the QA Review

The OMI Tasks Tracker is used to capture the results of the QA Review.

3.5.1 Access the DBI QA Review

To access the OMI Tasks Tracker, follow the instructions below:

1. Click Start > All Programs > Internet Explorer. The Internet Explorer window displays.

2. In the address bar, type http://omi. The OMI intranet window displays.

3. Select OMI Tasks Tracker from the menu located on the left side of the window.

4. Select QA below the OMI Tasks Tracker header.

3.5.2 To Complete a DBI Review

1. All of the information need to perform a DBI Review is reviewed for accuracy and timeliness. To complete the review, the QA Reviewer utilizes all of the tools used by the DBI analysts. These tools include the Contractor’s Database Integrity Procedures Manual, Case Classification Procedures Manual, DBI QA Checklist, DBI LOIs, Lexis CourtLink, etc. After the interview or file review is complete, capture the results in the OMI Tasks Tracker.

2. Select DBI Review from the QA Menu.

3. In the DBI Analyst field, select the analyst who performed the review from the drop-down list.

4. In the Attorney field, select the attorney whose cases were reviewed.

5. The system will automatically populate the Branch field. If it differs from the attorney’s current branch select the correct Branch from the drop-down list.

6. Select the Date of Interview from the drop-down list. The Date of Interview is the date the interview was conducted or the date the file review was conducted.

7. Select the Type of Interview from the drop-down list. There are two types of interviews, Interview and File Review.

8. Select the Date of Review using the calendar or enter a date. Date of Review is the first business day of the month being reviewed.

9. In the Number of Cases in CACL, the QA Reviewer counts and enters the total number of cases listed on the CACLs.

10. In the Number of Cases Updated, the QA Reviewer counts and enters the number of cases updated during the review.

11. In column under the DJ Number, enter “CACL”. In the DBI Flds field, enter 1. A CACL error can occur one of three ways. First, the DBI analyst is required to ask the attorney at the interview if there are any new cases not listed on the CACL. If the DBI analyst fails to ask the attorney this question, it is considered a CACL error. Second, the CACL should not be printed out by the DBI analyst prior to five business days of the interview. If the date on the CACL is greater than five days prior to the interview date, it is considered a CACL error. Third, the DBI analyst is required to verify that the attorney’s contact information on the first page of the CACL is correct. If it is incorrect, the DBI analyst must ask the attorney for the correct contact information. If the DBI analyst fails to ask the attorney this question, it is considered a CACL error.