HopScore: An Electronic Outcomes-Based Emergency Triage System

The need for effective triage is escalating, as crowded emergency departments (EDs) are challenged to safely manage excess patient demand with inadequate system capacity. Crowded EDs must maintain systems to quickly differentiate patients with critical conditions from those with less urgent needs. Inability to quickly identify significantly ill patients in crowded EDs contributes to delays in time-sensitive treatment and potentially avoidable deterioration, morbidity, and mortality. The “HopScore” tool was developed at Johns Hopkins to support objective triage decisions and improve patient differentiation based on outcomes data. Subsequently, the tool was embedded in the electronic medical record (EMR) under the name “E-triage,” and machine learning was used to provide acuity recommendations to triage nurses.

The objective of this study was to deploy E-triage and assess its uptake and impact on discerning clinical outcomes, timeliness of care, and triage nurse concordance. This project compared the E-triage to the Emergency Severity Index (ESI), the current standard for ED triage. Although ESI is used by 72 percent of EDs across the United States, it has several deficiencies that E-triage aims to overcome: (1) no link to critical patient outcomes, (2) a strong reliance on triage evaluators’ subjective judgment, and (3) poor discrimination of patients across the five-level index.

The specific aims of the project were as follows:

• Refine the HopScore decision tree algorithm using three independent population sources. 
• Design and develop HopScore as a software application (E-triage) that provides swift and interpretable triage decision-support embedded in the ED EMR at two clinical sites and as a free-standing, public-facing web application for general ED use. 
• Execute a phased implementation of the EMR-embedded HopScore (E-triage) application in clinical practice at both clinical sites. 
• Prospectively evaluate the HopScore (E-triage) application’s usability and impact on patient and operational outcomes compared to ESI. 

The project team deployed and prospectively evaluated E-triage, the machine-learning-based triage support tool that was developed based on HopScore. E-triage supported triage decision making by applying a machine learning algorithm to predict patients’ risk of acute outcomes and hospitalization. Nurses using E-triage could agree with or override the acuity recommendations generated by E-triage. Additionally, the nurses assigned an acuity level using ESI. Clinical outcomes and measures of timeliness were compared for patients triaged with ESI pre-implementation to those using E-triage post-implementation.

Compared to ESI, E-triage improved risk stratification for identifying low-risk patients, allowing providers to focus their time and attention on higher-severity patients. The time to decide whether to admit patients decreased by 58 minutes. Additionally, the overall time from arrival to seeing a provider was reduced by 10 minutes; however, the time from arrival to discharge was unchanged. Concordance of E-triage and nursing triage using ESI was 80 percent, supporting the value of combining E-triage with nurses’ clinical judgment. Harmonizing nurses’ clinical judgment with E-triage decision support improved risk stratification of patients at triage and operational performance. The team concluded that E-triage applied machine learning to distill large volumes of patient data to actionable information to support clinical decisions at the point of care.