An artificial intelligence approach to understanding trade-offs in emergency department decision-making

The National Institute of Nursing Research awarded Healthpartners Institute a project grant of $3,030,895 on July 23, 2026, to develop artificial intelligence tools that support emergency department decision-making, particularly triage nurse assessment and patient prioritization. The award funds research under the Nursing Research program (CFDA 93.361).

The project will create gold-standard emergency department decisions through expert annotation of patient cases across multiple decision-making tasks and establish standardized metrics for assessing decision-making performance. Healthpartners Institute will fine-tune and validate large language models on these annotated datasets, producing a model called "AI-Triage+" that will be tested against current human decision-making and existing pre-trained models. The work addresses a documented performance gap in current triage accuracy at approximately 60 percent compared to expert benchmarks, with the research targeting improvement through AI support during emergency department crowding and high cognitive load scenarios.

Performance of the work occurs in Bloomington, Minnesota, with a period of performance extending through June 30, 2030. Healthpartners Institute, a nonprofit research organization and subsidiary of Healthpartners, Inc., operates as a prime recipient of NIH awards and maintains research infrastructure anchored in clinical data access and patient population engagement across its integrated health system.

Abstract

Emergency department (ED) decision-making is inherently complex, influenced by uncertainty, time pressure, and high-stakes consequences. ED crowding exacerbates these challenges by increasing distractions and cognitive load, often leading to suboptimal decisions and adverse outcomes. Triage nurses are typically the first point of contact for medical needs assessment. For example, their role in prioritizing patients is one of the most consequential decisions, yet current triage accuracy is only about 60% compared to expert benchmarks. This performance GAP stems from the complexity of ED scenarios, which often exceed human cognitive limits. Artificial intelligence (AI), including large language models (llms), offer the potential to support triage nurses and all members of the ED team by improving decision-making on many ED tasks, especially during crowding. Advantages of such tools include rapid processing of large volumes of data, operating without fatigue, and being deployable on demand - all while rivaling human decision-making on a wide variety of tasks. With recent advances in AI, there is a timely opportunity to investigate its utility in supporting ED decision-making. While AI/llms could be vital support for ED decision-making, there is no gold standard (expert annotated labeled data) by which to evaluate the quality or accuracy ED decisions, and no standardized tasks or metrics. We propose to develop a set of gold-standard ED decisions derived by expert annotation on patient cases for a set of decision-making tasks, and establish a set of metrics for assessing decision-making performance. In addition, we will fine-tune and validate fine-tuned llms, with the best-performing model termed "AI-triage+". We will then test the performance of AI-triage+ on ED decision-making-tasks VS. Status QUO (human decisions) and existing pre-trained (generalist) AI models. Aim 1: compare the performance of AI models to a gold standard on five ED decision-making tasks: ESI (emergency severity index) level recommendation, patient-facing diagnostic question generation, acceptable safe wait time values for each patient, recommend diagnostic tests and ED procedures for the patient, and recommend ED resource level based on the patient's needs (AKA patient disposition). We will assess overall model performance and performance on specific subsets of cases, e.g., by diagnoses, age group, ED workload (overcrowding score). Aim 2: estimate the impact of AI models' decisions on clinical outcomes. Using existing risk equations, we will estimate how different ED decisions by AI models will impact ICU admissions or death in ED, hospital admissions, and length of hospital stay. We will also explore the potential of AI-triage+ to improve ED efficiency at a system level. This work directly supports the ninr focus on systems and models of care, offering foundational tools and insights to guide AI integration into ED practice.