The categorical conclusion scales that are used in the pattern comparison disciplines are not calibrated against the actual strength of the evidence

This federal Project Grant award, provided by the National Institute of Justice under the Forensics Training and Technical Assistance Program (CFDA 16.044), aims to calibrate a proposed conclusion scale for pattern comparison disciplines by computing consensus-based likelihood ratios. The $609,185 award to the Trustees of Indiana University, which runs from January 1, 2024 to December 31, 2026, has three primary deliverables:

1. Publish the image pairs, examiner response distributions, and corresponding consensus-based likelihood ratios from a black box study.
2. Provide the range of likelihood ratios associated with each statement in the categorical conclusion scale.
3. Develop a tool to allow examiners to compare casework image pairs against benchmark pairs with known likelihood ratios, enabling the computation of likelihood ratios for casework without requiring image pairs to leave the laboratory.

This work seeks to address the profound miscalibration in the language used to express conclusions in the friction ridge discipline, which has been found to overstate the strength of evidence by up to five orders of magnitude. By translating examiner responses into numerical likelihood ratios, the project aims to provide a transparent, appropriate, and consistent approach to communicating the strength of forensic evidence, in alignment with other scientific disciplines.

Recently we have established that the language used to express same-source conclusions in fingerprint comparisons overstate the strength of the evidence by up to five orders of magnitude (busey & coon, 2023). This is a profound miscalibration of the articulation language used to express conclusions in the friction ridge discipline. The goal of the present work is to calibrate a proposed conclusion scale by computing a set of consensus-based likelihood ratios, and then extend this to casework. Our approach translates responses by examiners in black box (error rate) studies to numerical likelihood ratios that describe the strength of support for the same- and different-source propositions. Once computed, these likelihood ratios establish the strength of support provided by each statement in a set of categorical conclusions. We will extend this approach to operational casework by validating as set of benchmark-based likelihood ratios, in which image pairs with known likelihood ratios are used as benchmarks against which a casework image pair can be compared. This will allow practitioners to include likelihood ratios in laboratory reports. We have three main deliverables. First, we will publish the image pairs, examiner response distributions and corresponding consensus-based likelihood ratios from the black box study. Second, we will provide the range of likelihood ratios that are associated with each statement in the categorical conclusion scale. Finally, we will provide a tool that will allow examiners to compare a casework image pair against a set of benchmark image pairs for which the likelihood ratios are known. Through interpolation we can compute a likelihood ratio for casework impressions without requiring images to leave the laboratory. These likelihood ratios can either supplement the categorical conclusions or even replace these statements. Since the likelihood ratio is not associated with knowledge of prior facts of the case, examiners can provide a value that does not subsume the role of fact-finder, directly communicates the strength support for the relevant propositions, and easily integrates with other facts of the case. CA/NCF current verbal conclusion scales are therefore becoming increasingly untenable because they appear to vastly overstate the strength of the evidence, are poorly defined, and understood differently by different people. Our approach leverages human expertise to provide a natural replacement that is transparent, appropriate, and consistent with other forensic scientific disciplines.