Integration of an NLP-based Application to Support Medication Management

Medication reconciliation (MedRec), the process of comparing a patient's medication order with all the medications the patient has been taking, is critical to preventing medication errors. However, it is challenging because patients see many clinicians who don’t share a common medication list. In addition, medication information frequently exists in free-text clinical notes and is not easily available for review.

This project builds on a previous AHRQ-funded project, Improving Outpatient Medication Lists Using Temporal Reasoning and Clinical Texts, that developed a natural language processing (NLP) system called the Medical Text Extraction, Reasoning, and Mapping System (MTERMS). MTERMS extracts and encodes medication information from electronic clinical notes into a structured format. That project showed that in patients with chronic diseases, 31 percent of the active medications mentioned in providers’ notes were missing from their medication lists. In this followup project, the investigators developed and evaluated an NLP-based tool to support medication list management at the point of care.

The specific aims of this project were as follows:

• Identify the requirements, use cases, workflow issues, and barriers to and facilitators of using clinical notes and NLP in the MedRec process using focus groups with providers and domain experts. 
• Design generic system architecture and an application that integrates an NLP system and a Web-based user interface within an existing MedRec system. 
• Pilot the tool in two primary care clinics and measure the utilization, usability, performance, and feasibility of the proposed methods and the tool in improving the process of MedRec in the outpatient setting using both quantitative and qualitative methods. 
• Distribute the methods and the tool so they are widely available to other researchers and health care institutions for non-commercial use. 

The investigators conducted a qualitative analysis to understand user requirements, use cases, system functional specifications, workflow issues, and barriers to and facilitators of using clinical notes for MedRec in the ambulatory setting. This information was used to design, develop, and implement a real-time Web-based tool called NotesLink, a Web application built upon the MTERMS NLP system. NotesLink presents NLP output and links to original notes to facilitate clinicians' review of medication discrepancies between notes and the medication list.

The investigators conducted a pilot study at two Brigham and Women’s Primary Care practices, measuring the performance and feasibility of the NotesLink system in improving MedRec. To evaluate the system, the investigators randomly selected two samples of patients. The first group was used for a comprehensive evaluation of NotesLink; the second group had at least one medication discrepancy identified by NotesLink and was used to evaluate the accuracy of the system. Results generated by the system were compared to a gold standard created by domain experts. Preliminary findings have shown that of 1,098 medication mentions found in 163 notes representing 306 unique medications, NotesLink achieved a precision of 70.1 percent and a recall of 73.7 percent in identifying all medication discrepancies and status. Fifty-two medication discrepancies were identified, of which 50 percent represented possible true discrepancies.

This work identified gaps and challenges of using advanced information technologies combining NLP and automated decision support to facilitate MedRec at the point of care and will inform future development of comprehensive medication reconciliation systems.