Health Information Technology Center for Education and Research on Therapeutics (Massachusetts)

Project Final Report (PDF, 564.63 KB) Disclaimer

Project Details - Ended

Project Categories

Summary:

The Centers for Education and Research on Therapeutics (CERTs) demonstration program is a national initiative to conduct research and provide education that advances the optimal use of therapeutics including drugs, medical devices, and biological products. The purpose of this health information technology (IT) CERT was to leverage health IT for three independent research projects focused on: 1) new technologies to improve pharmacosurveillance, 2) using new sources of data from clinical decision support (CDS) to identify physician-level variation, and 3) examining human factors principles in the design and implementation of medication-related CDS.

The specific aims of this project were as follows:

e-Pharmacovigilance II: Surveillance for Safety and Effectiveness

  • Develop a patient-reported, electronic health record (EHR)-integrated system to actively monitor the safety and effectiveness of treatment for patients taking Food and Drug Administration-approved medications for one of four common chronic conditions (diabetes, hypertension, insomnia, depression), with integrated management support by a pharmacist. 
  • Measure the reach, effectiveness, adoption, and implementation of this integrated module for adult primary care patients in the Brigham and Women's Primary Care Practice-Based Research Network. 

Physician-Level Variation in Medication Overrides of CDS 

  • Evaluate how much clustering of overrides there is by provider for both safety- and efficiency-related issues in the inpatient and outpatient settings. 
  • Evaluate the appropriateness of overrides overall and among those with high override rates. 
  • Characterize and understand the patterns and reasons for these override behaviors and decisions. 
  • Intervene with providers who have high override rates, for the specific categories that appear inappropriate. 

Examining Human Factors Principles in the Design and Implementation of Medication-Related Decision Support Alerts

  • Validate the human factors instrument on vendor and in-house medication-related decision support systems and determine how a "good" score on Instrument 1 correlates with override rates. 
  • Determine whether employing human factors principles can have an impact on physician behavior in terms of the rates of overriding alerts.
  • Evaluate user-related outcomes for systems that comply with human factors principles. 

The CERT established two cores that supported the three projects. The first core was the methodology and data resources core, which focused on coordinating study design and analytic strategies across research projects. The second, the translational and dissemination core, promoted interchange and cross-fertilization between projects, dissemination of research findings, and the translation of findings into practice through initiatives at the local, regional, and national levels.

Key findings are presented by project:

The e-Pharmacovigilance project conducted a cluster randomized trial at 18 clinics to test the efficacy of an EHR-linked, interactive voice response (IVR) tool that provided the support of a live pharmacist to detect patient-reported adverse drug symptoms. More than a third of patients reported symptoms and were transferred to the pharmacist, who assessed the severity and likelihood of symptoms being related to the medication. Chart reviews are in progress to identify the total number of physician-noted symptoms related to target medications, and to examine hypothesis that the IVR calls and pharmacist intervention would significantly increase the numbers of medication-related symptoms identified by physicians.

The Physician-Level Variation in Medication Overrides of CDS project evaluated the frequency and appropriateness with which physicians overrode alerts and the override reasons provided. The results indicated that more than half of the medication CDS alerts were overridden, with reasons varying depending on the type of alert. Many alerts that were overridden inappropriately had the potential to cause patient harm. The research team plans to use data collected from the project to identify physician-level variations and use the results to improve patient safety and efficiency.

The Human Factors Principles in Design and Implementation project used surveys and observational scenario-based analyses of clinical reasoning to assess how the design of alerts affects prescribing behavior impacting CDS acceptance and use. The study found a negative correlation between alert design and alert override rates, where alerts rated as better designs were less likely to be overridden. The survey showed that high alert volume was associated with higher likelihood of participants overriding alerts and not paying attention to even clinically significant alerts. Declining an alert suggestion was preceded by sometimes brief but often complex reasoning, prioritizing different aspects of care quality and safety, especially when the perceived risk was higher.

Health Information Technology Center for Education and Research on Therapeutics - 2012

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS11-004: Centers for Education and Research on Therapeutics (CERTs) (U19)
  • Grant Number: 
    U19 HS 021094
  • Project Period: 
    September 2011 – September 2016
  • AHRQ Funding Amount: 
    $4,115,767
  • PDF Version: 
    (PDF, 306.31 KB)

Summary: The Centers for Education and Research on Therapeutics (CERTs) demonstration program is a national initiative to conduct research and provide education to advance the optimal use of therapeutics (i.e., drugs, medical devices, and biological products). The program consists of six research centers and a CERTs Scientific Forum. The Health Information Technology CERT (HIT-CERT) is building on existing CERT resources and infrastructure and on Dr. Bates’ previous work related to health information technology (IT). The HIT-CERT is focusing research on issues related to health IT use and impact on therapeutics. In particular, the project work involves health IT research and its translation into clinical practice for pharmacosurveillance and medication-related clinical decision support.

The HIT-CERT will address therapeutics issues related to appropriateness, safety, and efficacy in the outpatient, inpatient, and transitions in care settings within the context of health IT. Two cores—the methodology and data resources core and the translation and dissemination core—oversee various aspects of all projects conducted within the HIT-CERT. The former core provides study design, data collection and management, and analysis support for all projects. The latter core promotes the dissemination of research findings and facilitates the translation of findings into practice and policy. Currently, the HIT-CERT has three projects underway for the 5-year duration of the Center.

  1. e-Pharmacovigilance II: Surveillance for Safety and Effectiveness. This project is developing and will implement an e-Pharmacovigilance system that integrates and interoperates with the Partners Healthcare electronic health record (EHR) system. The e-Pharmacovigilance system will interface with patients by an interactive voice response (IVR) system to monitor the safety and effectiveness of treatment with Food and Drug Administration-approved medications for common chronic conditions. This project is being conducted in three phases over 5 years. Phase 1 involves the development and pilot testing of the integrated pharmacovigilance system. Phase 2 is implementation of the system. Phase 3 involves assessment of the translation and dissemination of the system using the Reach, Effectiveness, Adoption, Implementation, and Maintenance or RE-AIM framework.
  2. Physician-Level Variation in Medication Overrides of Computerized Decision Support. This project is assessing, describing, and characterizing physician-level variations in response to computerized decision- support safety issues and efficiency suggestions for both the inpatient and outpatient settings. The project involves evaluating variations in medication overrides at baseline, followed by two cluster-randomized trials. A set of recommendations for improving decision support system safety and efficiency will be based on the findings.
  3. Examining Human Factors Principles in the Design and Implementation of Medication-Related Decision Support Alerts. This project will develop and evaluate a tool for assessing whether a medication-related decision support alert is appropriate relative to human factors principles. Over a 3-year period, an existing human factors instrument will be validated relative to override rates. Outcomes such as user’s satisfaction with alerts will be assessed. Additionally, a set of recommendations on the best way to design medication alerts will be developed. The HIT-CERT and its projects will provide information, strategies, and tools for utilizing health IT to improve clinical practices related to mediation safety, effectiveness, and cost.

Specific Aims:

  • Leverage new technologies to improve pharmacosurveillance. (Ongoing)
  • Use new sources of data from clinical decision support to identify physician-level variation and use these results to improve safety and efficiency. (Ongoing)
  • Improve medication-related clinical decision support. (Ongoing)

2012 Activities: Project development activities for each of the three projects continued during 2012. Activities for the e-Pharmacovigilance II: Surveillance for Safety and Effectiveness study included developing the IVR system; writing, testing, and programming a script; testing the IVR platform; and building a database for the IVR call and counseling protocol that will be part of the study. Additionally, Dr. Bates redesigned the study to be a randomized, controlled trial, as he feels this will strengthen the study. Institutional review board approval was received in December 2012, and the team planned to open the study to recruitment in early 2013.

The Physician-Level Variation in Medication Overrides of Computerized Decision Support project work for 2012 involved developing criteria for medication overrides and pilot testing the computerized decision support system. The project team also completed evaluating variations in medication overrides at baseline for the outpatient setting and is in the process of doing so for the inpatient setting.

Activities for the third project, Examining Human Factors Principles in the Design and Implementation of Medication-Related Decision Support Alerts, involved developing a study protocol, identifying and confirming study sites, and developing surveys. The project team confirmed nine study sites in total—six in the United States and three located internationally. Upon confirming participation in the study, the nine sites provided screen shots from their respective EHR systems, which will later allow the team to assess the compliance of the medication-related decision support alerts with human factors principles. Some sites also provided the team with their EHR alert logs.

Preliminary Impact and Findings: For the Physician-Level Variation in Medication Overrides of Computerized Decision Support study, the project team developed criteria for evaluating the appropriateness of medication overrides. The criteria demonstrate good reliability. The team also found that the rate of inappropriate overrides varied by the type of alert, with the highest rates being for renal substitutions, drug- drug interactions, and age-related substitutions. There was a much lower rate for duplicate suggestions, formulary substitutions, and patient allergies.

Preliminary findings for the Examining Human Factors Principles in the Design and Implementation of Medication-Related Decision Support Alerts project include learning that: 1) study sites had difficulty sharing the screen shots of their EHRs due to contractual obligations with their EHR vendors and the reluctance of the vendors to share the screenshots; and 2) many of the study sites were not able to produce alert logs from the EHR, indicating a gap in functionality offered by EHR vendor systems.

Target Population: Adults, Chronic Care*

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve the quality and safety of medication management via the integration and utilization of medication management systems and technologies.

Business Goal: Implementation and Use

* This target population is one of AHRQ’s priority populations.

Health Information Technology Center for Education and Research on Therapeutics - 2011

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS11-004: Centers for Education and Research on Therapeutics (CERTs) (U19)
  • Grant Number: 
    U19 HS 021094
  • Project Period: 
    September 2011 - September 2016
  • AHRQ Funding Amount: 
    $4,115,767
  • PDF Version: 
    (PDF, 197.81 KB)

Summary: The Centers for Education and Research on Therapeutics (CERTs) demonstration program is a national initiative to conduct research and provide education that advances the optimal use of therapeutics (i.e., drugs, medical devices, and biological products). The program consists of six research centers and a CERTs Scientific Forum. The Health Information Technology CERT (HIT-CERT) is building on existing CERT resources and infrastructure and on the investigators' previous work related to health information technology (IT). The HIT-CERT will focus research on issues related to health IT use and impact on therapeutics. In particular, the project work involves health IT research and its translation into clinical practice for pharmacosurveillance and medication-related clinical decision support.

The HIT-CERT will address therapeutics issues related to appropriateness, safety, and efficacy in the outpatient, inpatient, and transitions in care settings within the context of health IT. Two cores-the methodology and data resources core and the translation and dissemination core- will oversee various aspects of all projects conducted within the HIT-CERT. The former core will coordinate the study for all projects by providing study design, data collection and management, and analysis support. The latter core will promote the dissemination of research findings and facilitate the translation of findings into practice and policy.

Currently, three projects are planned for HIT-CERT's 5-year duration:

  1. e-Pharmacovigilance II: Surveillance for Safety and Effectiveness. This project will develop and implement an e-Pharmacovigilance system that integrates and interoperates with the Partners Healthcare electronic health record system. The e-Pharmacovigilance system will interface with patients either by a Web-based portal or an interactive voice response system to monitor the safety and effectiveness of treatment with Food and Drug Administration-approved medications for common chronic conditions. This project will be conducted in three phases over 5 years. Phase 1 involves the development and pilot testing of the integrated pharmacovigilance system. Phase 2 is implementation of the system. Phase 3 involves assessment of the translation and dissemination of the system using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework.
  2. Physician-Level Variation in Medication Overrides of Computerized Decision Support. This project will assess, describe, and characterize physician-level variations in response to computerized decision support safety issues and efficiency suggestions for both the inpatient and outpatient settings. The project will include evaluations of variation in medication overrides at baseline, followed by two cluster randomized trials. A set of recommendations for improving decision support system safety and efficiency will be developed based on the findings.
  3. Examining Human Factors Principles in the Design and Implementation of Medication-Related Decision Support Alerts. This project will develop and evaluate a tool for assessing whether a medication related decision support alert is appropriate relative to human factors principles. Over a 2-year period, an existing human factors instrument will be validated relative to override rates. Outcomes such as user's satisfaction with alerts will be assessed. Additionally, a set of recommendations on the best way to design medication alerts will be developed.

In the wake of increasing implementation and use of health IT and electronic prescribing, the HIT-CERT and its projects will provide information, strategies, and tools for utilizing health IT to improve clinical practices related to mediation safety, effectiveness, and cost.

Specific Aims:

  • Leverage new technologies to improve pharmacosurveillance. (Ongoing)
  • Use new sources of data from clinical decision support to identify physician-level variation and use these results to improve safety and efficiency. (Ongoing)
  • Improve medication-related clinical decision support. (Ongoing)

2011 Activities: Much of the HIT-CERT program efforts in 2011 focused on project start-up. All three projects have received approval from the Brigham and Women's Hospital's Institutional Review Board. Project development activities for each project are in progress and data collection will be initiated in 2012.

Preliminary Impact and Findings: The project has no findings to date.

Target Population: Adults, Chronic Care*

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve the quality and safety of medication management via the integration and utilization of medication management systems and technologies.

Business Goal: Implementation and Use

* This target population is one of AHRQ's priority populations.

Health Information Technology Center for Education and Research on Therapeutics - Final Report

Citation:
Bates D. Health Information Technology Center for Education and Research on Therapeutics - Final Report. (Prepared by Brigham and Women's Hospital under Grant No. U19 HS021094. Rockville, MD: Agency for Healthcare Research and Quality, 2017. (PDF, 564.63 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
Population: 
Medical Condition: 

An International Evaluation of User Perceptions of Drug-Drug and Drug-Allergy Interaction Alerts

Citation:
Robertson A, Neri P, Burdick E, et al. An International Evaluation of User Perceptions of Drug-Drug and Drug-Allergy Interaction Alerts. American Medical Informatics Association (AMIA) 2014 Annual Symposium, Washington DC, Nov 2014. (PDF, 522.04 KB)
(Persons using assistive technology may not be able to fully access information in this report. For assistance, please contact Corey Mackison)
Principal Investigator: 
Population: 
Medical Condition: 

An International Evaluation of Drug-Drug Interaction Alerts That Should be Non-Interruptive in U.K. and U.S. Settings.

Citation:
Slight SP, Seger DL, Thomas S, et al. An International Evaluation of Drug-Drug Interaction Alerts That Should be Non-Interruptive in U.K. and U.S. Settings. American Medical Informatics Association (AMIA) 2013 Annual Symposium, Washington, DC, US. Nov 2013. (PDF, 445.45 KB)
(Persons using assistive technology may not be able to fully access information in this report. For assistance, please contact Corey Mackison)
Principal Investigator: 
Document Type: 
Population: 
Medical Condition: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.