Health Information Technology Hazard Manager (Pennsylvania)

Project Final Report (PDF, 990.63 KB)

Project Details - Ended

Project Categories

Summary:

There is growing recognition that the use of health information technology (IT) carries risks that can compromise patient safety and lead to harm. Any characteristic of a health IT application that compromises patient care processes or causes harm is referred to as a 'hazard.' These hazards typically are analyzed retrospectively, after harm has been identified, but ideally should be identified before harm has occurred. The health IT hazard manager (Hazard Manager), developed by researchers at Geisinger Health System, is a tool designed to support the characterization of hazards and communicate their potential and actual causality in adverse effects. This project extensively tested a beta version of the Hazard Manager, revised the tool, and evaluated the final tool.

The specific aims of this project were to:

  • Design, build, and test the Hazard Manager software.
  • Beta test the Hazard Manager software in six to eight study sites.
  • Refine the ontology based on findings from the Beta test.
  • Deliver a fully-tested and refined version of the Hazard Manager Software and final report.

The ECRI Patient Safety Institute worked with the project team to program and operate a Web-based beta version of the Hazard Manager. Seven study sites—hospitals and health systems—participated in the beta test, as did five health IT vendors, and representatives from the Agency for Healthcare Research and Quality (AHRQ), the Food and Drug Administration, and the Office of the National Coordinator for Health IT. The beta test included analysis of the 495 hazards entered by study site participants; qualitative data collection concerning usefulness and usability of the tool; inter-rater reliability testing of participants' interpretations of standardized hazard scenarios; and a project summary meeting at AHRQ headquarters. Following evaluation, the features of the tool deemed to be the most important included: a clear focus on how hazards are discovered; a thorough explanation of the many causes that alone or in combination lead to health IT hazards; information about the impact of the hazards; and detailed information about the urgency of hazard correction and the steps taken to correct or mitigate a hazard.

In contemplating the possible deployment of the Hazard Manager as part of a National infrastructure for monitoring and improving health IT safety, the team outlined the need for the following aspects: 1) data aggregation at multiple levels; 2) version control; 3) confidentiality; and 4) access to detailed information. In addition, options to implement such a program were outlined, including the expansion of the Health IT Common Formats to include proactive health IT hazard identification with aggregation and reporting through the National Patient Safety Database.

Health Information Technology Hazard Manager - 2012

Summary Highlights

  • Principal Investigator: 
  • Organization: 
  • Funding Mechanism: 
    Accelerating Change and Transformation in Organizations and Networks (ACTION)
  • Contract Number: 
    290-2006-00011I-14
  • Grant Number: 
    August 2010 – May 2012
  • AHRQ Funding Amount: 
    $763,135
  • PDF Version: 
    (PDF, 245.42 KB)

Summary: This project focused on developing and testing a software tool called the Health Information Technology Hazard Manager. The goal of the software is to enable providers to classify and communicate hazards related to the use of electronic health records (EHRs) and other health information technology (IT) so that problems can be fixed or controlled before they cause patient harm. An example of a hazard would be entering an order for the wrong patient, which could be caused by user interface or the absence of an automated patient-identity confirmation.

Rather than looking retrospectively at accidents or near-misses, the Hazard Manager was designed to collect structured information about hazards associated with specific health IT products. The tool collects information about four main components of hazards: discovery, causation, impact, and corrective action. The system collects information on the nature of the hazard, its cause, and how it was corrected. The Hazard Manager can help health care providers assemble consistent and organized information about the potential hazards identified in their IT products, as reported by other users of the same products. When deployed regionally or nationally, health care organizations will benefit from a mechanism to categorize, manage, and resolve hazards consistently, and to anticipate hazards that others have encountered in the next upgrade of their IT products. The tool will also allow health IT vendors to view hazards their customers have identified and prioritize fixes for future upgrades. The Hazard Manager contains three levels of security: 1) participating health care organizations can enter and see information about its own hazards and those reported by unidentified others who use the same products; 2) vendors can view hazards reported by their unidentified customers; and 3) health care organizations, vendors, policymakers, and researchers can view aggregated, unidentified reports of all hazards.

A Beta test was conducted under the auspices of a patient-safety organization (PSO). Beta test participants entered several hazards per week for 6 months. They also entered hazard scenarios (vignettes) to test interrater differences. The software was evaluated on usability and usefulness and refined accordingly, based on group and individual discussion with participating health care organizations, their software vendors, and federal policymakers.

Project Objectives:

  • Design, build, and test the Hazard Manager software. (Achieved)
  • Beta test the Hazard Manager software in six to eight study sites. (Achieved)
  • Refine the ontology based on findings from the Beta test. (Achieved)
  • Deliver a fully tested and refined version of the Hazard Manager software and final report. (Achieved)

2012 Activities: The project was originally scheduled to be completed in February 2012, but due to delays in obtaining signatures on the PSO agreements and receiving the institutional review board waiver, the contract was extended by 3 months and the project was completed in May 2012. The focus of activity during the year was on refining the ontology based on findings from the Beta test and developing the final report, (PDF, 991 KB).

Impact and Findings: Several discoveries resulted from the Beta testing of the Hazard Manager, such as finding that an individual’s role determines the types of hazards that come to their awareness. For example, the IT implementation teams learn about potential hazards during testing, while patient-safety teams may learn about care process compromises during their review of patient care. Hospitals have separate IT and patient incident reporting systems that, while not explicitly designed for hazard identification, can help teams identify hazards.

The tool specifies whether software design flaws are related to usability, data quality, or software specifications. In terms of impact on patients, the software captures the severity, duration, and type of harm—focusing not only on physical but also raising awareness about psychological, financial, and reputational harm. The most important features of the Hazard Manager include:

  • A clear focus on how hazards are discovered, including the point in the health IT lifecycle in which a hazard is identified, how it is discovered, and how information about a hazard is shared
    within and beyond a care delivery organization.
  • Explication of the many causes that alone or in combination lead to health IT hazards, including distinct software design flaws and the absence of effective IT protections to help users avoid
    errors.
  • Information about the impact of hazards as well as the type, severity, and duration of patient harm. For hazards that have not yet caused harm, the Hazard Manager supports estimation of
    potential for harm, including the number of patients who could be affected and the likelihood that an alert user would notice the hazard before a patient was harmed.
  • Detailed information about the urgency of hazard correction and the steps taken to correct or mitigate a hazard. The departments that need to approve a hazard control plan, and the
    departments responsible for carrying out that plan, can also be recorded.

Target Population: General

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to support patient-centered care, the coordination of care across transitions in care settings, and the use of electronic exchange of health information to improve quality of care.

Business Goal: Knowledge Creation

Health IT Hazard Manager - 2011

Summary Highlights

  • Principal Investigator: 
  • Organization: 
  • Funding Mechanism: 
    Accelerating Change and Transformation in Organizations and Networks (ACTION)
  • Contract Number: 
    290-2006-00011I-14
  • Project Period: 
    August 2010 - May 2012
  • AHRQ Funding Amount: 
    $763,135
  • PDF Version: 
    (PDF, 177.49 KB)

Summary: This project focused on developing and testing a software tool called the Health IT Hazard Manager. The goal of the Hazard Manager is to enable providers to classify and communicate hazards related to the use of electronic health records (EHRs) and other health information technology (IT) so that problems can be fixed or controlled before they cause patient harm. An example of a hazard would be entering an order for the wrong patient, which could be due to the user interface or the absence of an automated patient identity confirmation.

Rather than looking retrospectively at accidents or near-misses, the Hazard Manager is designed to collect structured information about hazards associated with specific health IT products. The Hazard Manager collects information about four main components of hazards: discovery, causation, impact, and corrective action. The system collects information on the nature of the hazard, its cause, and how it was corrected. The Hazard Manager can help health care providers assemble consistent and organized information about the potential hazards identified in their IT products, as reported by other users of the same products. When deployed regionally or nationally, health care organizations will benefit from a mechanism to consistently categorize, manage, and resolve hazards, and understand hazards others have encountered in the next upgrade of their IT products. The Hazard Manager will also allow health IT vendors to view hazards their customers have identified and prioritize fixes for future upgrades. The Hazard Manager contains three levels of security: 1) participating health care organizations can enter and see information about its own hazards and those reported by unidentified others who use the same products; 2) vendors can view hazards reported by their unidentified customers; and 3) health care organizations, vendors, policymakers, and researchers can view aggregated, unidentified reports of all hazards.

The Beta test was conducted under the auspices of a patient safety organization (PSO). Beta test participants entered several hazards per week for 6 months. They also entered hazard scenarios (vignettes) to test inter-rater differences. The Hazard Manager was evaluated on usability and usefulness and refined accordingly, based on group and individual discussion with participating health care organizations, their software vendors, and federal policymakers.

Project Objectives:

  • Design, build, and test the Hazard Manager software. (Achieved)
  • Beta test the Hazard Manager software in six to eight study sites. (Achieved)
  • Refine the ontology based on findings from the Beta test. (Achieved)
  • Deliver a fully-tested and refined version of the Hazard Manager software and final report. (Ongoing)

2011 Activities: The primary focus of activities in the first 9 months of the project was on designing, planning, and programming of the Hazard Manager software, finalizing PSO agreements with participating health care organizations, and obtaining a review waiver from the institutional review board (IRB). Beta testing began with seven sites and Beta test data entry included nearly 500 actual hazards. Data analysis was completed in December 2011. Due to delays in obtaining signatures on the PSO agreements and receiving the IRB waiver, the contract was extended by 3 months with a new project end date of May 2012.

Preliminary Impact and Findings: There were several discoveries that resulted from the Beta testing of the Hazard Manager, such as finding that an individual's role determines the types of hazards they become aware of. For example, the IT implementation teams learn about potential hazards during testing, while patient safety teams may learn about care process compromises during their review of patient care. Hospitals have separate IT and patient incident reporting systems that, while not explicitly designed for hazard identification, can help teams identify hazards.

Failure to control hazards are often labeled as "user error;" the Hazard Manager focuses on the missing safeguards in IT systems that fail to protect users from making mistakes. Hazards are often labeled "software design flaws;" the Hazard Manager specifies whether these flaws are related to usability, data quality, or software specifications. In terms of impact on patients, the Hazard Manager captures the severity, duration, and type of harm-focusing not only on physical harm but also raising awareness about psychological, financial, and reputational harm.

Target Population: General

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

Health IT hazard manager beta-test: appendix F – “other (specify)” entries

Citation:
Walker JM, Hassol A, Bradshaw B, et al. Health IT hazard manager beta-test: appendix F – “other (specify)” entries. (Prepared by Abt Associates and Geisinger Health System, under Contract No. 290-06-00011I-14). Rockville, MD: Agency for Health Care Research and Quality, 2012. AHRQ Publication No. 12-0058-EF. (PDF, 66.86 KB)
Principal Investigator: 

Health IT Hazard Manager Beta-Test - Final Report

Citation:
Walker JM, Hassol A, Bradshaw B, et al. Health IT Hazard Manager Beta-Test - Final Report. (Prepared by Abt Associates and Geisinger Health System, under Contract No. 290-06-00011I-14). Rockville, MD: Agency for Health Care Research and Quality, 2012. AHRQ Publication No. 12-0058-EF. (PDF, 990.63 KB)
Principal Investigator: 
Document Type: 
Research Method: 

Health IT hazard manager beta-test: appendix A

Citation:
Walker JM, Hassol A, Bradshaw B, et al. Health IT hazard manager beta-test: appendix A. (Prepared by Abt Associates and Geisinger Health System, under Contract No. 290-06-00011I-14). Rockville, MD: Agency for Health Care Research and Quality, 2012. AHRQ Publication No. 12-0058-EF. (PDF, 1.6 MB)
Principal Investigator: 

Health IT hazard manager beta-test: appendix B - revised hazard manager ontology

Citation:
Walker JM, Hassol A, Bradshaw B, et al. Health IT hazard manager beta-test: appendix B - revised hazard manager ontology. (Prepared by Abt Associates and Geisinger Health System, under Contract No. 290-06-00011I-14). Rockville, MD: Agency for Health Care Research and Quality, 2012. AHRQ Publication No. 12-0058-EF. (PDF, 57.08 KB)
Principal Investigator: 

Health IT hazard manager beta-test: appendix C - descriptive analysis

Citation:
Walker JM, Hassol A, Bradshaw B, et al. Health IT hazard manager beta-test: appendix C - descriptive analysis. (Prepared by Abt Associates and Geisinger Health System, under Contract No. 290-06-00011I-14). Rockville, MD: Agency for Health Care Research and Quality, 2012. AHRQ Publication No. 12-0058-EF. (PDF, 235.08 KB)
Principal Investigator: 

Health IT hazard manager beta-test: appendix D – relational analysis results

Citation:
Walker JM, Hassol A, Bradshaw B, et al. Health IT hazard manager beta-test: appendix D – relational analysis results. (Prepared by Abt Associates and Geisinger Health System, under Contract No. 290-06-00011I-14). Rockville, MD: Agency for Health Care Research and Quality, 2012. AHRQ Publication No. 12-0058-EF. (PDF, 51.39 KB)
Principal Investigator: 

Health IT hazard manager beta-test: appendix E – inter-rater cognitive testing results

Citation:
Walker JM, Hassol A, Bradshaw B, et al. Health IT hazard manager beta-test: appendix E – inter-rater cognitive testing results. (Prepared by Abt Associates and Geisinger Health System, under Contract No. 290-06-00011I-14). Rockville, MD: Agency for Health Care Research and Quality, 2012. AHRQ Publication No. 12-0058-EF. (PDF, 84.63 KB)
Principal Investigator: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.