Attachment_(9)_Centralized_records_Management_QA_Review_Procedures.doc

Centralized Records Management

Quality Assurance Review Procedures Prepared For:

Department of Justice

Civil Division

Office of Assistant Attorney General

FOIA, Records & E-Discovery Office Information Systems & Networks Corporation 10411 Motor City Dr.

Suite 750

Bethesda, MD 20817

March 09, 2007 Centralized Records Management

QA Review Procedures

Table of Contents iTable of Contents iiiList of Figures

11.0 Introduction

11.1 Program Background

21.2 File Organization

41.3 Quality Assurance Program

72.0 QA Methodology

72.1 Formal Reviews

82.1.1 Sample Selection

82.1.2 Evaluation Procedures

92.1.3 Error Identification

92.1.4 Error Rates

92.1.5 Quality Control

102.2 Informal Reviews

113.0 ARU QA Review

133.1 The ARTS ARU Room Printout

143.2 ARU File Maintenance QA Review

153.2.1 Selecting the Sample

153.2.2 Reviewing the Sample for Accuracy

333.3 ARU File Section QA Review

333.3.1 Selecting the Sample

333.3.2 Reviewing the Sample for Accuracy

403.4 ARU File Check Out Log QA Review

433.4.1 Selecting the Sample

433.4.2 Reviewing the Sample for Accuracy

443.4.3 Reporting the Results

463.5 Correction Verification

534.0 RMU QA Review

534.1 The ARTS RMU Printout

534.2 RMU File Maintenance QA Review

544.2.1 Selecting the Sample

554.2.2 Reviewing the Sample for Accuracy

695.0 FRC QA Review

725.1 QA Review of Case Files Retired to FRC

735.1.1 Reviewing the Accession for Accuracy

855.2 Sending File Materials to the FRC

866.0 Reporting Procedures

866.1 Historical Reports

876.2 Monthly Status Report

886.3 Fiscal Year Report

1Appendix A - Random Sample Selection

2Excel Data Analysis Tools

12Research Randomizer

17Randomizer Ransom Sample Selection Program

35Random Number Tables

44Ordinal Position Sampling

1Appendix B – DJ Number Order / General Numbers

2DJ Number Order

8General Numbers

9Identifying “General Numbers”.

10Handling of “General Number” Files.

1Appendix C – DOJ File Jacket and Accordion File Samples

1Appendix D – Key Order Numbering of File Sections

1Appendix E – Category Definitions

1Appendix F – Completed QA Review Samples

1Appendix G – Sample Fiscal Year Report

1Appendix H – Guide to the ARTS Database (To Be Published)

List of Figures

3Figure 1.1 DOJ Case File Jacket (OBD-155)

Figure 3.1 ARU File Unit Codes and Locations Figure 3.2 ARTS ARU Room Printout Sample Figure 3.3 OBD-155 DOJ Case File Jacket Figure 3.4 Checked Out Files sheet Figure 3.5 Case Disposition / Closure Form Figure 3.6 Substitute Case Closure Form Figure 3.7 Sample “NEW” Document Figure 3.8 Substitute NEW Figure 3.9 Missing File Section Check List Figure 3.10 ARU File Maintenance Review – Error Detail Report Figure 3.11 ARU File Maintenance QA Review - Summary Page Figure 3.12 ARU File Section Review – Error Detail Report Figure 3.13 ARU File Section QA Review – Summary Page Figure 3.14 ARU File Check Out Log Figure 3.15 Checked Out Files Sheet Figure 3.16 ARU File Check Out Review – Error Detail Report Figure 3.17 ARU File Check Out Review – Summary Page Figure 3.18 Correction Verification Review Summary Page – File Maintenance Figure 3.19 Correction Verification Review Summary Page – File Section Figure 3.20 Correction Verification Review Summary Page – File Check Out Figure 3.21 Correction Verification Review Detail Form Figure 4.1 RMU File Maintenance QA Review – Error Detail Report Figure 4.2 RMU File Maintenance QA Review – Summary Page Figure 4.3 RMU File Section Request Log Figure 4.4 RMU Process of Requesting File Sections from the FRC Figure 4.5 RMU Request for File Section Review - Summary Page Figure 4.6 RMU Request for File Section QA Review – Error Detail Report Figure 5.1 File Section Pull List Figure 5.2 FRC Box Labeling Diagram Figure 5.3 ARTS Automated Box List Figure 5.4 Records Transmittal and Receipt SF-135 Figure 5.5 FRC Retirement QA Review – Error Detail Report Figure 5.6 FRC Retirement QA Review – Summary Page Figure 6.1 Review History Report Figure 6.2 Monthly Status Report

1.0 INTRODUCTION

Information Systems & Networks Corporation (ISN) provides quality assurance (QA) services to the Civil Division of the Department of Justice (DOJ) according to the terms and conditions of contract DJJ13-C-2482. These procedures are for QA of the Centralized Records Management services provided to the Office of Assistant Attorney General, FOIA, Records & E-Discovery Office (OAAG – FREDO) by another Government contractor. The Centralized Records Management Program ensures:

· The maintenance, retirement and retrieval of Division records;

· The compliance with regulatory requirements for the maintenance and disposition of records;

· The standardization of the handling of these records;

· The efficient use of file storage space; and

· The effectiveness and efficiency with which the Division can locate and retrieve needed files.

Under the direction of the Division's Records Management Officer in the Office of Policy and Management Operations, the employees of the OAAG - FREDO records management contractor administer the following:

· The maintenance, storage and control of open active litigation case files in close proximity to the attorneys who use them at the Active Records Units (ARUs); and

· The maintenance, storage and control of closed litigation case files at the Records Management Unit (RMU) and the retirement of those files to the Federal Records Center (FRC).

1.1 Program Background

To track the location and content of litigation case files, the Civil Division developed the Automated Records Tracking System (ARTS) in 2000. This system is linked to the Case Management System (CASES) so that when a new case is classified and a Department of Justice control number (DJ Number) is created, it is simultaneously available within ARTS.

The ARUs store open active litigation case files in close proximity to the attorneys who make frequent and extensive use of them. After litigation is concluded and the attorneys close the cases, files are transferred to the RMU for temporary off-site storage. Files that have been closed for at least one year are eligible for shipment to the FRC for long-term storage. At any of these locations, continued requests for filing or retrieval of files are expected.

Numerous procedures and tasks are required to receive, process, maintain, transfer, retire and retrieve the case file sections stored at the ARUs and at the RMU, plus the additional case file sections stored at the FRC. Each task must be performed accurately, effectively and efficiently to ensure that information is retrieved when needed by the attorneys and to ensure compliance with the requirements of the National Archives and Records Administration.

1.2 File Organization

A Department of Justice Civil Division litigation case is identified by a unique control number known as a Department of Justice (DJ) Number. The DJ Number is assigned when a new case is classified. It normally consists of the following three segments separated by a hyphen:

1) A classification code indicating the subject area of the case which generally relates to a specific statute or issue type (over 200 codes in use);

2) A code number for the judicial district of the case; and

3) A sequential number.

A Civil Division litigation case file is the total collection of all information for a specific DJ Number and consists of one or more sections. The Department requires that the first section always be in an OBD-155, the official Department of Justice case file jacket (Figure 1.1). All other sections may be in an OBD-155 or in legal size accordion file folders. A case file may include from one to several hundred sections.

Figure 1.1 DOJ Case File Jacket (OBD-155)

Case documents are organized within 28 sequentially numbered categories, for example, the General (00) category, the Correspondence (03) category and the Pleadings/Motions (13) category. A detailed definition of all 28 categories is provided in Appendix E. Each category may contain as many physical sections as required which are sequentially numbered within the category, for example, 03.0001, 03.0002, etc. All file sections are stored in open library shelving units. The current location of each section is recorded and tracked in ARTS.

A set of labels is affixed to each section to identify and track case files. A bar code label displays the DJ Number, Category Number, Section Number, Key Order Number and Location Code in both alpha-numeric and machine readable code. The Title Label displays the DJ Number, case caption, section description and Key Order Number. (Older Title Labels display the case caption and section description only). The labels are affixed to all sections as illustrated in Figure 1.1. An additional label displaying the DJ Number is affixed to the upper left hand corner of the back side of each section.

1.3 Quality Assurance Program

Numerous procedures and tasks are involved in the creation, processing, maintenance, transfer, retirement and retrieval of case files within the Centralized Records Management Program. Accuracy in performing each task is essential to ensure that attorneys can locate information when needed and to ensure compliance with the requirements of records management statutes and regulations. To ensure that the Centralized Records Management services provided to OAAG - FREDO meet contract quality requirements, the Civil Division has instituted a Quality Assurance program which uses an independent, third party contractor to review contract deliverables and processes on a systematic basis.

The QA program assesses the accuracy of contractor performance in following procedures and performing tasks regarding the receipt, processing, maintenance, transfer, retirement and retrieval of case files. This manual describes the QA review process of the contractor's Centralized Records Management Program performance and is organized as follows:

· Section 2 Sampling Methodology, describes ISN’s method of population identification, sample selection and sample collection.

· Section 3 ARU QA Review Procedures, provides detailed QA review procedures for all ARU activities and forms used to assess and record the accuracy of the selected sample items.

· Section 4 RMU QA Review Procedures, provides detailed QA review procedures for all RMU activities and forms used to assess and record the accuracy of the selected sample items.

· Section 5 FRC QA Review Procedures, provides detailed QA review procedures and forms for performing an FRC Retirement review.

· Section 6 Reporting Procedures, contains narrative descriptions of the historical, monthly and fiscal year reports provided to OAAG - FREDO, with examples.

· Appendix A, Random Sample Selection, contains instructions for the methods used to obtain a statistically valid sample for use in the QA Review process. These methods are

a. Excel Data Analysis Tools – uses Excel spread sheet statistical tools,

b. Research Randomizer - an internet web site,

c. Randomizer Random Sample Selection Program,

d. Random Number Tables, and

e. Ordinal Position Sampling – uses every 20th number starting with a randomly chosen beginning sample.

· Appendix B, DJ Number Order / General Numbers, provides an explanation of the Department of Justice’s procedures for assigning DJ Numbers to a case and an explanation of General Number files.

· Appendix C, DOJ Case File Jacket and Accordion File Samples, provides an illustration of the case file jackets and accordion files used to file case material.

· Appendix D, Key Order Numbering of File Sections, provides an explanation of the Department of Justice’s procedures for assigning Key Order Numbers to case file sections.

· Appendix E, Category Definitions, provides a list and definitions of the 28 categories used to file case materials.

· Appendix F, Completed QA Review Samples, provides examples of completed ARU, RMU and FRC QA Reviews.

· Appendix G, Sample Fiscal Year Report, provides an example of the annual Fiscal Year Report submitted to OAAG - FREDO.

· Appendix H, Guide to the ARTS Database, provides an explanation of how to use the database. (To be published)

2.0 QA Methodology

ISN quality assurance (QA) methodology is based on distinguishing between "formal" and "informal" reviews. This distinction depends on whether or not a random sample will yield statistically valid characteristics of the population and whether or not an accuracy rate calculation is required to meet Department of Justice (DOJ) acceptance standards. ISN conducts formal reviews from random samples of populations. Entire populations are sampled for informal reviews.

2.1 Formal Reviews

It is generally accepted statistical procedure that a sample size of 5% is valid for populations of the sizes described in the contract, as long as all of the following criteria are met:

· The population must be a collection of similar units.

· The sample selection method must be random.

· The evaluation procedures must be objective and consistent within each review and within each series of reviews that are compared.

· The error calculation procedures must be objective and consistent within each review and within each series of reviews that are compared.

· The accuracy rate computation method must be objective and consistent within each review and within each series of reviews that are compared.

ISN’s formal review methodology meets all of the above criteria and includes the following:

· The selection of a statistically valid random sample;

· An analysis of individual items using specifically defined evaluation procedures;

· The determination of error rate percentages; and

· A Quality Control review.

Each of the above components is more fully described in Sections 3, 4 and 5.

2.1.1 Sample Selection

ISN uses one of three methods to select sample items for evaluation:

· Computer generated random samples (Microsoft Excel Data Analysis Tools, Randomizer.org website or Randomizer Random Sample Selection Program);

· Published random number tables; or

· Ordinal position of the item within the population.

The selection method used depends on the attributes of the population, the type of product being reviewed and the point in a process at which the sample is drawn. Appendix A provides instructions for the various methods of random sample generation currently used by ISN.

2.1.2 Evaluation Procedures

ISN has developed evaluation procedures for all QA reviews based on the following premises:

· All requirements in the contract must be met.

· Errors that are identified must be classified in a manner that is standardized, objective and accurate.

· The reporting of results to DOJ must be informative so that the data can be used to identify specific errors to be corrected and to help identify error trends.

The procedures for each type of review are based on a common methodology of error identification and error rate calculation.

2.1.3 Error Identification

The QA Reviewer examines individual sample DJ Numbers and assesses whether or not conditions conform to the handling and processing requirements for the each item. As the item is examined, the QA Reviewer completes detailed review forms and indicates for that item:

· An unique identifier

· Errors (if existing)

When the QA Reviewer is not sure that a condition is an error, the discrepancy is identified, the circumstances are explained and supporting documentation that illustrates the error in question is provided to OAAG - FREDO for further evaluation.

2.1.4 Error Rates

After each sample item in the population has been evaluated and the results recorded, the QA Reviewer completes a summary page review form using the Microsoft Excel report template. The report template automatically calculates and records error rates for each evaluation criterion on the summary page as described in Sections 3, 4 and 5. Separate error rates and values are calculated for each criterion examined.

2.1.5 Quality Control

The QA Reviewer submits the completed review forms, source documents and supporting documentation to the QA Program Manager for quality control review. The QA Program Manager ensures that all work products produced by the QA Reviewer are complete and accurate. All report forms and supporting documentation must be present and all errors or discrepancies clearly substantiated. The completed review is submitted to OAAG - FREDO for review.

2.2 Informal Reviews

ISN may also perform a number of informal reviews that do not include determining error rate percentages. These reviews are for the purpose of evaluating whether populations are complete, verifying stages in processing pipelines or subjectively reviewing work product quality. Informal reviews are performed upon OAAG - FREDO request. Examples of work products reviewed informally are:

· Sequential DJ Number checks,

· Corrections to QA reports or implementation of corrective action following QA reports, and

· ARU category special surveys.

3.0 ARU QA Review

Seventeen Active Records Units (ARUs) currently operate in the downtown Washington area. Each ARU is assigned and referred to by its unique file unit code. The current file unit codes, locations and room numbers are provided in Figure 3.l.

ARU LOCATIONS

Figure 3.1 ARU File Unit Codes and Locations

The clerk in each ARU is responsible for:

· Updating the ARTS database: adding and deleting sections, transferring cases, checking sections in and out of the ARU;

· Maintaining the files: adding documents to existing sections and creating new sections as necessary;

· Retrieving files when requested;

· Replacing checked out and missing sections with the proper out-guide;

· Re-filing materials upon check in; and

· Retiring cases to the appropriate repository.

Attorneys check file sections out of an ARU by entering their name, date of check out, DJ Number and the Key Order Number(s) of all files borrowed into the Check Out Logbook located in each ARU. When litigation concludes and the cases are closed, the files are shipped to the Civil Division’s Records Management Unit (RMU) for temporary off-site storage.

The location of each section of every case is tracked in the Automated Records Tracking System (ARTS). The ARTS system contains a complete history of each section starting with the day it was created and tracks each check out, check in and/or transfer to another location. When an ARU is assigned for QA review, ISN is provided with a printout from ARTS that contains the complete inventory of cases currently housed in the assigned ARU. The printout is organized by DJ Number and lists all sections located in the ARU as well as the current status of each section and the date of closure for closed cases. This printout is used to select the samples for review. It also serves as a tool for verifying the current disposition of the selected samples against the ARU shelf. As the ARTS database is a live database with multiple changes occurring daily all disagreements between the ARU printout and the conditions found on the shelf are confirmed via the ARTS database before penalties are assessed.

ISN reviews the accuracy of the ARU files maintained by the OAAG - FREDO records management contractor. The sections that follow describe ISN’s File Maintenance, File Section and File Check Out reviews.

3.1 The ARTS ARU Room Printout

The ARU room printout (Figure 3.2) is generated by the OAAG - FREDO client and must be obtained prior to performing the review. This printout is used to itemize and select the samples for the ARU File Maintenance and File Section QA Reviews and is the tool used to verify the current disposition of the selected samples against the ARU shelf.

Figure 3.2 ARTS ARU Room Printout Sample

3.2 ARU File Maintenance QA Review

The purpose of the ARU File Maintenance Review is to verify the accuracy of the ARTS database and to confirm that the sections are properly labeled and filed in the correct sequence. Each sampled DJ Number is assessed for the following criteria:

ARU File Maintenance QA Reviews are normally completed in approximately two week intervals. The review results are submitted to OAAG - FREDO when complete for approval or revision, if necessary. The results often determine the frequency an ARU is reviewed and may demonstrate the need for a special review of the room.

3.2.1 Selecting the Sample

A sample selection of 5% of the population (the total number of DJ Numbers in the ARU) is identified from the ARTS ARU room printout. The sample selected for the File Maintenance Review must be randomly generated and independent of the sample selected for the File Section Review. See Appendix A for specific instructions regarding random sample generation.

3.2.2 Reviewing the Sample for Accuracy

The focus of the File Maintenance Review is the organization and labeling of the files on the ARU shelves. The files should be in the proper order and each file should be labeled following established guidelines. The QA Reviewer makes note of any discrepancies between the ARTS report and what is found on the shelf making sure to confirm each discrepancy against the ARTS database, as changes may have occurred after the printout was produced for the review. As long as conditions are substantiated in the database, no error is recorded.

Note: Discrepancies normally reported as part of the File Section Review are noted without penalty.

3.2.2.1 Population Definitions

Each criteria examined in the File Maintenance Review evaluates specific populations. The populations of the File Maintenance Review are defined as follows:

1. Number of DJ’s: The total number of cases in the sample. This number is used to calculate the error rate for criteria A and C.

2. Closed Cases: The number of cases in the sample that have a Case Closed Date on the ARTS room printout. This number is used to calculate the error rate for criterion J.

3. In First File Sections: The total number of first file sections (00.0001) in the sample that are checked in and on the shelf. This number is used to calculate the error rate for criteria B and K. With one exception (category 25 only cases), every case must have a 00.0001 section. If the case contains only category 25 sections the rules that apply to section 00.0001 are applied to 25.0001.

4. Disposition/Closure Forms: The total number of Disposition/Closure forms found in the first file section (00.0001) of each case in the sample. This number is used to calculate the error rate for criterion I. Do not count closure forms dated before 1995 or substitute closure forms. These are not evaluated during the review.

5. Total Sections: The total number of all sections for the sampled DJ Numbers. This number is used to calculate the error rate for criterion L.

6. IN Sections: The total number of sections in the sampled cases that are not represented by Out-guides. This number is used to calculate the error rate for criteria D, G, H and M.

Note: IN Sections = Total Sections – (Red Out-guides + Blue Out-guides)

7. Red Out-guides: The total number of checked out sections in the sampled cases. This number is used to calculate the error rate for criteria E and F.

8. Blue Out-guides: The total number of sections in the sampled cases that have been replaced with a Blue Out-guide and/or have a non-availability code in ARTS. This number is used to calculate criterion N.

3.2.2.2 Criteria Review Procedures

The following section provides a description of each evaluation criterion of the ARU File Maintenance Review with guidelines for reviewing the accuracy of each criterion.

1. Criterion A - DJ Numbers are in proper numeric sequence on the shelf. The QA Reviewer verifies that the sampled case files are filed in sequential order by DJ Number. Once the file is located on the shelf the DJ Number before and after the file is compared to the ARTS room printout to verify that the case file has been filed correctly.

Note: Appendix B contains specific guidelines for the proper filing of DJ Numbers. The QA report does not always print the DJ Numbers in the proper sequence as defined in the appendix. Any violation of the guidelines listed in the appendix is assessed an error.

2. Criterion B – The first file section is in the proper DOJ case file jacket. The first file section is required to be filed in an official DOJ case file jacket, the OBD-155. Figure 3.3 illustrates an OBD-155.

Figure 3.3 OBD-155 DOJ Case File Jacket

The QA Reviewer verifies that the first file section is filed within an official DOJ case file jacket (OBD-155) as shown above. If the first section is found in an accordion file an error is assessed.

Exception: Class 201 first file sections may be filed in an accordion file.

3. Criterion C – The DJ Number sections are filed in sequence by Key Order Number. The QA Reviewer verifies that all sections within each sampled DJ Number are filed in correct Key Order sequence. Refer to Appendix D for specific details on the Key Order Numbering system.

4. Criterion D – Sections listed on ARTS as checked in are on the shelf. This criterion reports as errors any file section that has been removed from the ARU without being updated in ARTS. This criterion only applies to the sections that appear on the room printout as checked in. The QA Reviewer verifies that all the files listed on the ARTS printout as checked in are present on the shelf. If a checked in file is not located on the shelf, the QA Reviewer must verify the discrepancy against the ARTS database before assessing an error.

5. Criterion E – Sections listed on ARTS as checked out are not on the shelf. The QA Reviewer verifies that all checked out sections are off the shelf. Any section that has been returned to the shelf without being updated in ARTS is recorded as an error. All discrepancies are verified in ARTS before errors are assessed, as the section may have been updated after the room printout was generated.

6. Criterion F – Checked out sections are replaced with a Red Out-guide on the shelf. Each file section borrowed from an ARU must be recorded in the ARTS database as checked out. After the system is updated, ARTS generates a Checked Out Files sheet (Figure 3.4) which is inserted into a red file folder and placed on the shelf in the exact location of the borrowed file section. The QA Reviewer verifies that all sections identified as checked out on the ARTS room printout have been replaced with Red Out-guides on the shelf. Any checked out section that has not been replaced by a Red Out-guide is assessed as an error.

CHECKED OUT FILES

When you check out files, YOU ARE PERSONALLY RESPONSIBLE FOR THEIR RETURN to the ARU. We need to know when you transmit files to others – PLEASE TELL US. Otherwise, the Division will continue to hold YOU accountable for their return.

The following file sections were checked out on:

12/27/00 13:38:04

To: Doe, John Code: DJOE

Office: NPB

NPB8063

From this file room: P1 8009S Natl Place

DJNO: 157-99-9999

Caption: ACB Company v US

Category

Section Number

Section Description

00 General

00.0001

Figure 3.4 Checked Out Files sheet

7. Criterion G – Sections have all necessary labels and the information on the labels is correct and legible. The QA Reviewer verifies that all sections contain legible and accurate ARTS labels. Any section missing one of the three required labels (DJ Number, Title and Bar Code), containing a label with incorrect information (such as the wrong Key Order Number for the section) or has an illegible label is assessed an error. Special attention is paid to the room location bar code label, as this can change frequently.

8. Criterion H – Sections have properly attached labels. The QA Reviewer verifies that all labels have been affixed to the files in the correct position. Any section with labels affixed in the wrong place on the file is marked as an error. Refer to Appendix C for proper label placement.

Criterion I – Disposition/Closure forms in case files have a Case Closing Unit (CCU) stamp. Disposition/Closure forms (Figure 3.5) are filed in the first section of a case file. Some cases do not have a Disposition/Closure form and some may have multiple forms. The QA Reviewer verifies that each Disposition/Closure form found within a case file contains a CCU stamp. This stamp confirms that the form has been correctly processed through the Case Closure Unit. Each form without a CCU stamp is assessed as an error unless it is dated prior to 1995. Substitute closure forms (Figure 3.6) are not evaluated.

Figure 3.5 Case Disposition / Closure Form Substitute Case Closure Form

DJ Number: 39-11-7892 Date of Closure: 05/06/2004

Caption: Verline, Vladimir v INS

Disposition:

Supreme Court:

Date:

Appellate Court: Judgment Affirmed

Date: 04/28/2004

Trial Court:

Date:

Monetary: To Government To Opposition

Total Award $0.00 $0.00

Relator $0.00

Restitution $0.00

Equitable/Nonmonetary No No

Attorney: Friedman, Nancy F

Reviewer: Evans, Richard M

Date Printed: 11/19/2004

Figure 3.6 Substitute Case Closure Form

9. Criterion J – Cases closed on the ARTS printout have been transferred from the ARU. If the sampled DJ Number has a ‘Case Closed’ date on the ARU printout, the case files should be transferred to the RMU within one month of that date. If the case file is still located in the ARU and more than one month has elapsed from the Case Closed date, an error is assessed.

· Criterion Ja – Number of closed cases without checkouts: The QA Reviewer reports the number of J errors caused by closed cases found in the ARU that have no sections checked out. Error rate calculations are not provided for this criterion.

· Criterion Jb - Number of closed cases with checkouts: The QA Reviewer reports the number of J errors caused by closed cases found in the ARU that have sections checked. Error rate calculations are not provided for this criterion.

Exceptions:

· If a closed case is delegated, it may remain on the shelf up to one year past the Case Closed date. The QA Reviewer must verify the case’s delegated status in the CASES database before errors are assessed.

· If a special request has been issued to hold a case past the closure date, the following procedure applies:

1. A Blue Out-guide is placed on the shelf with a note explaining that the case will be held until a specified date;

2. A “Hold Until” date is entered in ARTS;

3. A comment is entered in the case notes section of ARTS providing an explanation for the hold status and the length of the hold.

If any of these procedures have not been followed, a criterion J error is assessed.

· Special Media may be retained on the shelf longer than one month for closed cases. Currently, there is no set amount of time the special media section may remain in an ARU after the closure of the case.

10. Criterion K – The first file section contains the “NEW” or “Substitute NEW”: The “NEW” document (i.e., the original document which initiates the start of the case is stamped with NEW in red ink) or a “Substitute NEW” form must be filed in the first section of each case file. Figure 3.7 illustrates a “NEW” document. If a document with the “NEW” stamp or a “Substitute NEW” form (Figure 3.8) is not located within the first section, an error is assessed.

Figure 3.7 Sample “NEW” Document

_ SUBSTITUTE NEW _

Washington, D.C. 20530

Date: 1/1/2001

MEMORANDUM FOR DJ FILE, NUMBER: 123-321-5556

Active Records Unit (ARU): P1 The attorney assigned to this case has not submitted to the ARU the document stamped “NEW”, which is necessary to create the official case file for the above numbered case.

Mary M. Montgomery

(the assigned attorney) could not provide The NEW for the following reason She cannot find the document.

The ARU Records Technician has advised the assigned attorney to submit the NEW to the ARU whenever the NEW becomes available.

This memorandum will substitute for the NEW as authority for jacketing documentary materials concerning this case.

By : John Jones Supervisor, Active Records Unit

ARU Technician :

Sam Smith Date : 1/1/2001

When Original NEW Acquired and Filed, Remove Substitute NEW

Figure 3.8 Substitute NEW

Exceptions:

· Class 189 cases do not have a “NEW”. This is a special class dealing with requests from foreign courts asking for assistance with litigation against persons residing in the U.S.

· General Number cases are not required to have “NEWs” or “Substitute NEWs”. Most General Number files should not have “NEWs”.

11. Criterion L – All sections located within the ARU are on the ARTS room printout. The QA Reviewer verifies that all sections and out-guides for each sampled case file located within the room appear on the ARTS printout. Any section or out-guide found in the room that is not on the ARTS printout is marked as an error, unless the section was added or transferred after the printout was generated. All discrepancies are confirmed in ARTS before errors are assessed.

· Criterion La – Number of sections not in ARTS: The QA Reviewer reports the number of L errors caused by sections that are not in the ARTS database. This includes sections or out-guides on the shelf that were deleted from the database and unlabeled sections that have not been added into the database. If an out-guide represents more than one section, each section is assessed as an La error. Error rate calculations are not provided for this criterion.

· Criterion Lb - Number of sections located in another unit on ARTS: The QA Reviewer reports the number of L errors caused by sections found in the ARU even though the ARTS database shows it to be at another location. If an out-guide representing more than one section is found, each section is counted as an Lb error. Error rate calculations are not provided for this criterion.

12. Criterion M – Each file section has the appropriate DJ, Category, Section and Key Order Number handwritten on the folder. The QA Reviewer verifies that each file section has the appropriate DJ, Category, Section and Key Order Number handwritten on the front of the file jacket or accordion file. The following items are verified:

· All numbers must be written on the file exactly as they appear on the label.

· All numbers must be written directly on the file jacket or accordion file (not on a label).

· The numbers must be written in the upper left corner on the front of the file jacket and the left side of the back tab of the accordion file. Refer to Appendix C for the proper location.

· All written numbers must be completely visible and not covered by a label.

13. Criterion N – Missing sections have been identified in ARTS and replaced with Blue Out-guides. Each file section missing from an ARU must be recorded in the ARTS database as missing. After the system is updated a “Missing File Checklist” (Figure 3.9) is inserted into a blue file folder and placed on the shelf in the exact location of the missing file section. The QA Reviewer verifies that each missing file section in the sample has been identified as such in ARTS and a Blue Out-guide has replaced the file section on the shelf. If a missing file has been replaced with a Blue Out-guide but the non-availability code has not been entered into ARTS an error is assessed. If a missing file has been found but the non-availability code has not been removed from ARTS an error is assessed.

Missing File Section Checklist

DJNO

Key Order Section Number

Date Discovered Missing

Assigned Attorney

Have Completed the Following Search:

Search

Date Completed

1. All: Checked ARTS for current location of the section (to see if the section was checked out or transferred to another location)

2. All: Checked shelves for possible misfiling (such as transposition of numbers, e.g., 157-65-1234 instead of 157-56-1234 or misfiling by year, e.g., 154-96-123 filed after 154-98-122)

3. All: Reviewed all file sections in the shelving unit and the shelving units on each side of that shelving unit

4. All: Contacted the last person to have checked out the file section (to verify accuracy of ARTS check-in record)

5. All: Consulted ARTS to see of additional sections had been added to the case (to determine if the missing section had been changed into a new, subsequently added section)

6. ARUs: Contacted other ARUs (to see if the section was inadvertently filed in those ARUs)

7. ARUs: Contacted the RMU (to see if the case had been sent to the RMU without proper data-entry on ARTS to reflect the transfer)

8. ARUs: Contacted all staff listed in timekeeping records as having worked on the case (to see if they know the whereabouts of the file section)

9. ARUs: Contacted the assigned attorney (if not included in 8, above)

10. RMU: Contacted originating ARU to see if file nay still be there (possible inaccurate transfer record in ARTS)

11. RMU: Searched all work and storage areas, including special media storage

12. RMU: Researched relevant FRC transfer records for the DJNO for possible errors in Shipment or data-entry

13. All: Discussed search with supervisor to obtain further search instructions

14. All: Entered appropriate “missing code” in ARTS

Code Entered: _______

Additional Comments:

Name_____________________ Date_______________

Figure 3.9 Missing File Section Checklist

3.2.2.3 Category 23 Procedures

Category 23 sections are maintained exclusively in ARU N1. These sections contain privileged information and are handled differently from other sections. Category 23 sections are sealed envelopes containing documents. Labels are affixed and file information is written on the envelopes in the exact same manner as accordion folders and DOJ jackets. The category 23 section remains in N1 when the case closes When ARU N1 is reviewed, the OAAG - FREDO client provides the QA Reviewer with two printouts. One printout lists every section in the room (including category 23) and the other lists only category 23 sections. Category 23 sections are reviewed using a modified version of the File Maintenance Review. Criteria B, I, J and K do not apply to category 23 files and are not evaluated for the Category 23 review. All of the other criteria are evaluated exactly the same.

3.2.2.4 Reporting the Results

The QA Reviewer records individual errors by DJ Number on the Error Detail Report of the ARU File Maintenance Review (Figure 3.10). The DJ Number, Key Order Number, Attorney ID, Category, Section and number of errors are entered for each DJ Number containing an error. All of the information entered into the Error Detail Report is available on the QA room printout EXCEPT Attorney IDs. The Attorney name is found by searching ARTS by the DJ Number and opening the Case Detail screen.

ARU File Maintenance Review Error Detail Report

Room:

DJ Number Key Order #

Atty ID

Field

Cat.

Sec.

# Err

Comments

Figure 3.10 ARU File Maintenance Review – Error Detail Report

For each review criterion, the sample size, number in error and error rate calculation is provided on the ARU File Maintenance Review summary page (Figure 3.11). Figure 3.11 also illustrates the error calculation formulas for each criterion. The population definitions for each criterion are provided at the top of the page. The completed review and supporting documentation are submitted to the OAAG - FREDO client for review, and if necessary, correction. Copies of the entire review and supporting documentation are maintained at ISN.

ARU FILE MAINTENANCE REVIEW

QA Report Date:

Evaluator:

ARU:

Date of ARTS Printout:

Number of DJs: W

In First File Sections: V Disposition/Closure Forms: U Closed Cases: T

Total Sections: X In Sections: Y Red Outs : Z Blue Outs: S Field

Sample Size

No. In Error

% In Error

Criterion

A

W a a/W

DJ Numbers are out of proper numeric sequence.

B

V b b/V

First file section is not in proper DOJ case file jacket.

C

W c c/W

DJ Numbers with sections out of proper Key Order Number sequence.

D

Y d d/Y

Sections listed on ARTS as checked in and are not on the shelf.

E

Z e e/Z

Sections listed on ARTS as checked out and are on the shelf.

F

Z f f/Z

Checked out sections not properly replaced by Red Out-guides.

G

Y g g/Y

Sections with missing labels or have labels with incorrect or illegible information.

H

Y h h/Y

Sections with labels that are not properly attached.

I

U i i/U

Disposition/Closure forms that do not have the CCU stamp.

J

T j j/T

Cases closed on the ARTS printout that have not been transferred from the ARU.

Ja

-- Number of closed cases without checkouts.

Jb

-- Number of closed cases with checkouts.

K

V k k/V

First file section does not contain the “NEW” or “Substitute NEW”.

L

X l l/X

Number of sections found on the shelves that are not on the ARTS printout.

La

-- Number of sections not on ARTS.

Lb

-- Number of sections located in another unit on ARTS.

M

Y m m/Y

Sections that do not have DJ, Category, Section and Key Order Numbers written on the file.

N

S n n/S

Number of known missing sections where proper procedures have not been followed.

FOR EACH QA CRITERIA: %IN ERROR = NUMBER IN ERROR X 100

SAMPLE SIZE

Figure 3.11 ARU File Maintenance QA Review - Summary Page

3.3 ARU File Section QA Review

The purpose of the ARU File Section Review is to assess the accuracy of the filing of documents within each file section. Documents must be properly filed by DJ and Key Order Number. The following criteria are evaluated in the ARU File Section Review:

3.3.1 Selecting the Sample

A sample selection of 5% of the population (the total number of DJ Numbers in the ARU) is identified from the ARTS ARU room printout. The sample selected for the File Section Review must be randomly generated and independent of the sample selected for the File Maintenance Review. See Appendix A for specific instructions regarding random sample generation.

3.3.2 Reviewing the Sample for Accuracy

The focus of the File Section Review is the organization and condition of the documents within the files. The documents should be filed by the correct DJ Number according to the Key Order Number. Reverse chronological order should be followed for the documents in categories 00, 03 and 13. The QA Reviewer notes any discrepancies for later research in the ARTS database before assessing errors. If discrepancies are not justified by updates to the ARTS database an error is assessed.

The QA Reviewer locates the sampled DJ Number and compares the shelf conditions to the ARTS room printout. Notes are made of any discrepancies between the ARTS report and what is found on the shelf for later confirmation of each discrepancy against the ARTS database as changes may have occurred after the printout was produced for the review. As long as conditions are substantiated in the database no error is recorded. If the majority of the sections in a sample is checked out or missing from the shelf, the DJ Number before or after the sample is substituted.

Note: Discrepancies normally reported as part of the File Maintenance Review are noted without penalty.

3.3.2.1 Population Definitions

The criteria examined in the File Section Review evaluate the accuracy of four specific populations. The populations of the File Section Review are defined as follows:

1. Total Number of Sections: The total number of sections reviewed in the sampled cases excluding sections that have been replaced with Out-guides. This number is used to calculate the error rate for criteria A, B and C.

2. Number of 00/03/13 Sections: The number of category 00, 03, and 13 sections reviewed within the total number of sections. This number is used to calculate the error rate for criterion D.

3. First File Sections: The total number of first file sections that are not represented by Out-guides. This number is used to calculate the error rate for criterion E.

4. First File Sections with Disposition/Closure Forms: The number of first file sections that contain at least one Disposition/Closure Form. This number is used to calculate the error rate for criterion F.

3.3.2.2 Criteria Review Procedures

The following section provides a description of each evaluation criterion of the ARU File Section Review with guidelines for reviewing the accuracy of each criterion.

1. Criterion A – Documents are properly filed by DJ Number. Every section for each sampled DJ Number is reviewed to ensure that the documents within each section have been filed in the appropriate DJ Number case file. Every document should be labeled on the first page with a DJ Number and category. The QA Reviewer examines every original document within each file to verify that the DJ Number appearing on the document matches the DJ Number for the case that it is filed in. For example, if a document labeled 39-28-100 is found within a section of case 39-22-100, an error is assessed.

Exception: Attorneys may keep photocopies of documents from other cases as reference material. Xerox copies of other case documents are excluded from this review. If the DJ Number written on the document is in ink (not Xeroxed), it is considered an original document.

Note: If the document is not labeled with the DJ Number and category another identifying feature must be used. The case caption or the case number assigned by the trial court may be used for identification. The “NEW” should provide an alternative means of identification for the documents in the file.

2. Criterion B – Documents are filed correctly by Key Order Number. The QA Reviewer examines every document within every file for each DJ Number sampled. The following items are evaluated:

· The documents are correctly filed by Key Order Number. The last segment of the Key Order Number is disregarded when reviewing the accuracy of this criterion. For example, a document labeled with Key Order number 03.0015.0002 must be in a 03.0015 file, either 03.0015.0001 or 03.0015.0002 is acceptable. If the document is not filed correctly by the Key Order Number an error is assessed. See Appendix D for specific Key Order Number information.

· All documents are filed in the correct category. For example, if a category 03 file contains a document labeled for category 13, an error is assessed.

Exception: Attorneys may use copies of documents from other cases and/or sections as reference material. Errors are only assessed for original documents that have been incorrectly filed.

3. Criterion C – Documents contained in the file sections are organized in a ‘readable’ orientation. The documents within every section for each DJ Number sampled are reviewed. An error is assessed for any section containing documents filed in a backwards, upside down, or sideways orientation, or for a section containing documents which are crumpled. All documents should be in the same orientation within the file.

Exception: When documents within a file have been organized and tabbed in the file jacket or are inside a manila folder/three ring binder, they have been organized by the attorney. These files are excluded from review of the document orientation.

4. Criterion D - Documents in the General (00), Correspondence (03) and Pleadings/Motions (13) sections are filed in reverse chronological order. Only the 00, 03 and 13 categories are reviewed for this criterion. Every document in these categories is reviewed to ensure they are filed in reverse chronological order. The most recent document should be the first in the file. The only documents that do not follow the reverse chronological order rule are Disposition/Closure Forms and “NEWs”. The proper filing of those documents is covered in criteria E and F.

Exception: When documents within a file have been organized and tabbed in the file jacket or are inside a manila folder/three ring binder, they have been organized by the attorney. These files are excluded from the review of proper chronological order.

5. Criterion E – The “NEW”/”Substitute NEW” is at the back of the file. The “NEW” document should always be the first document placed in the file, i.e. at the bottom of the file. If there are any documents filed behind the “NEW”, a criterion E error is assessed.

6. Criterion F – The Disposition/Closure form is the top document in the file. The Disposition/Closure form should always be the last document placed in a file, i.e. on the top of file. If there are any documents filed over the form, a criterion F error is assessed.

3.3.2.3 Reporting the Results

The QA Reviewer records individual errors by DJ Number on the Error Detail Report of the ARU File Section Review (Figure 3.12). The DJ Number, Key Order Number, Attorney ID, Category, Section and number of errors are entered for each DJ Number containing an error.