Attachment_(3)_Accuracy_Standards.doc

Automated Litigation Support

QA Review Procedures

Appendix D – Contract Accuracy Standards

Contract Accuracy Standards

For Coding, Keying, OCR and Database Accuracy

I.

Accuracy: Definition and Formula

Subject to the conditions stated below, the Contractor shall ensure that the document coding and data reduction tasks are performed at a level of quality such that:

1.

all recorded information is accurate;

2.

all relevant information is recorded; and, 3.

no irrelevant information is recorded.

"Accuracy" shall be determined on the basis of a character count. The formula for determining the accuracy of each batch of document coding or its associated data tape shall be:

A - (B + C + D) X 100

A where:

A = all characters which should be recorded

B = all recorded characters which are inaccurate

C = all relevant characters which are not recorded

D = all irrelevant characters which are recorded.

As the Statement of Work provides, the COR shall determine for each Task Order the size of the coding/keying "batch." For example, assume the document coding forms associated with the documents from microfilm reels ABC 001 through ABC 003 should contain 10,000 characters (in the fields other than Document Number and Document Date). If the coding batch in this example had 10 relevant characters which were inaccurately coded ("B" in the formula above), 5 relevant characters which were omitted ("C") and 6 irrelevant characters recorded ("D"), the accuracy of the coding batch would be:

10,000 - (10 + 5 + 6)

10,000

X 100 = 99.79%

This coding batch would be rejected on the basis of the accuracy requirements and definition of "characters" given in Section II and III below:

II.

Document Coding: Definition of "Characters"

For the purposes of bill and determination of coding accuracy, characters shall be counted as illustrated below:

EXAMPLE 1:

(Document Date field)

*DT

760101

6 characters

EXAMPLE 2:

(Document Title, illustrating a variable length text field)

REPORT ON EXPENDITURES

20 characters

EXAMPLE 3:

(Document Type field, illustrating "check-offs" for valid field types)

*TY

(LET) ; MEM ; REP ; (CHR)

2 characters

Only the coded characters, as defined and illustrated above, are counted.

For litigation document databases, the Document Number and Document Date fields must be 100% accurate. A single error in either of these fields shall constitute grounds for rejection of the entire batch. The minimum acceptance standard for the remaining fields will be 99.95% based on a character count, as defined and illustrated above.

For all databases other than litigation document database, the COTR will define the field which constitutes the "unique identifying record number" and any other primary sort fields (for instance "Deposition Date" in the Case Management System). For each of these databases, the "unique identifying record number" and the other primary sort fields (no more than two other sort fields) must be 100% accurate. A single error in such a field constitutes grounds for rejection of the entire batch. The minimum acceptance standard for the remaining fields will be 99.95% based on a count of characters.

All determinations of accuracy will be made by comparing the document coding forms with the associated source documents for a statistically valid random sampling of documents from the batch. Coding deliverables will not be considered accepted until 30 days after the successful loading of all DCFs in a given batch, including correction of errors detected during the loading process.

[Note: The contractual standards for accuracy are provided above. However, in practice, the current acceptance standards are 98.5% accuracy for Key Fields and 95% accuracy for Other Fields.]

III. Keying/Key Verification: Definition of "Characters"

For the purposes of billing and determination of data reduction accuracy, characters shall be counted as illustrated below:

(Document Date Field)

*DT

760101

1 + 1(tab) + 6 + 1 (enter)

9 characters

EXAMPLE 2:

(Author field, illustrating all sub fields completed)

*AU

JONES,LP

ABC MINING

l + l(tab) + 8 + 1(DELIM.) + 10 + l (enter) = 22 characters

EXAMPLE 3:

(Title field)

*TI

REPORT ON EXPENDITURES

1 + 1(tab)

+ 1 (enter)

20 characters

EXAMPLE 4:

(Document Type field, illustrating "check-offs" for valid field types)

*TY

LET ; CHR

1 + 1(tab) 3 + 1(delim.) 3 + 1(enter)

10 characters

Only the keyed characters, as defined and illustrated above, are counted; key verification characters are not counted.

For litigation document databases, the Document Number and Document Date fields must be 100% accurate. A single error in either of these fields shall constitute grounds for rejection of the entire batch. The minimum acceptance standard for the remaining fields will be 99.95% based on a character count, as defined and illustrated above.

For all databases other than litigation document database, the COTR will define the field which constitutes the "unique identifying record number" and any other primary sort fields (for instance "Deposition Date" in the Case Management System). For each of these databases, the "unique identifying record number" and the other primary sort fields (no more than two other sort fields) must be 100% accurate. A single error in such a field constitutes grounds for rejection of the entire batch. The minimum acceptance standard for the remaining fields will be 99.95% based on a count of characters.

All determinations of accuracy will be made by comparing the keyed data with the associated document coding forms for a statistically valid random sampling of document coding forms from the batch. Keying deliverables will not be considered accepted until 30 days after the successful loading of all DCFs in a given batch, including correction of errors detected during the loading process.

[Note: The contractual standards for accuracy are provided above. However, in practice, the current acceptance standards are 98.5% accuracy for Key Fields and 95% accuracy for Other Fields.]

IV.

Overall Database Accuracy Standards

For purposes of determining overall database accuracy with respect to the documents, characters shall be counted as illustrated below:

EXAMPLE 1: (Document Type Field)

MEM

3 Characters

CHR

3 Characters

6 Characters

EXAMPLE 2: (Author Field)

ANDREWS,J

SAS

l2 Character

Only characters of data present in the database are counted; tags, "repeat" numbers, trailing blanks, etc. are not counted.

For litigation document databases, the Document Number and Document Date fields must be 100% accurate. A single error in either of these fields shall constitute grounds for rejection of the entire batch. The minimum acceptance standard for the remaining fields, when compared with the information in the documents themselves, shall be 99.90% based on a character count, as illustrated above.

For databases other than litigation document databases, the COTR will define the fields which constitute the "unique identifying record number" and the other primary sort fields (no more than two other sort fields). For each of these databases, the "unique identifying record number" and other primary sort fields must be 100% accurate. The minimum acceptance standard for the remaining fields will be 99.90% based on a character count, as illustrated above.

All determinations of accuracy will be made by comparing printouts of the database records with a statistically valid sampling of the hard copies of the documents which they represent. Coding, keying, tape loading, and error correction deliverables will not be considered accepted until 30 days after the successful loading of all DCFs in a given batch, including correction of errors detected during the loading process.

[Note: The contractual standards for accuracy are provided above. However, in practice, the current acceptance standards are 98.5% accuracy for Key Fields and 95% accuracy for Other Fields.]

V.

Accuracy of OCRed Text

For purposes of calculating the accuracy of OCRed text, characters and characters in error will be counted as per the following examples:

[Original Transcript Text] more if they look like it would be complicated. In the

Total character count = 55.

[OCRed Text For Same Line] more hey look like it would be cohplicated. In.the

Total missing characters = 3

Total incorrect characters = 1

Total unnecessary/irrelevant characters = 1.

Accuracy will be measured by visually comparing OCR output against the original page. Accuracy for this line would be calculated as follows:

55 – (3 + 1 + 1)

55 x 100 = 90.9%

Notes:

Extra spaces between words in the OCRed text are not counted as errors. However, an extra space in the middle of a word is counted as an error.

Special characters inserted by the OCRing device to flag potential OCRing errors are counted as erroneous characters.