Implementing and Improving the Integration of Decision Support into Outpatient Clinical Workflow (Indiana)

Project Final Report (PDF, 468.03 KB)

Project Details - Ended

Project Categories

Summary:

This project analyzed a clinical decision support (CDS) tool for colorectal cancer screening that was integrated into an ambulatory clinical workflow. In the first phase of the study the project team collected qualitative data via site visits, interviews, and direct observation in order to analyze the factors needed for effective integration into clinical workflow in different electronic medical records (EMRs).  Findings from the first phase informed the second phase of the project in which alternative designs were prototyped in a simulated setting in order to test their impact on workload, efficiency, and usability.  The participants in the study included the Regenstrief Institute, the Veteran’s Health Administration (VHA), and Partners Healthcare System. The main objectives of the project were to:

  • Identify key approaches to CDS development for colorectal cancer screening at two VHA Medical Center sites and two nationally recognized non-VHA sites to obtain effective CDS integration into clinical workflow. 
  • Develop and test CDS design alternatives for improved integration into clinical workflow through a controlled simulation study and subsequent implementation.

Based on findings from phase one, the project team developed prototyped design enhancements to the VHA’s colorectal cancer (CRC) screening clinical reminder. In the controlled simulation experiment, 12 primary care providers were given four simulated patient encounters using both current and redesigned CRC screening reminders. “Think aloud” techniques were used during the scenarios.  A usability survey, workload assessment, and workflow integration surveys were completed. 

Despite each site using different EMRs and CDS, the project team found that barriers encountered were similar. These barriers included: a lack of coordination among “outside” exam results, primary care, and specialty care; poor data organization and presentation; omission of provider and patient education in the decision support tool; lack of interface flexibility; the need for technological enhancements; unclear role assignments; organizational issues; and a disconnect between decision support and quality reporting.

The enhancements made to the VHA reminder showed positive impact on usability and workflow integration, but not on workload.  The qualitative portion of the study showed broad support for the enhancements.  Specific suggestions were collected for future improvements to the tool.  The project team concluded that new CDS prototypes are needed to improve data organization and presentation; integrate outside results; and provide just-in-time education and cognitive support.

Implementing and Improving the Integration of Decision Support into Outpatient Clinical Workflow - 2011

Summary Highlights

  • Principal Investigator: 
  • Organization: 
  • Contract Number: 
    290-06-0013-3
  • Project Period: 
    September 2007 - March 2011
  • AHRQ Funding Amount: 
    $394,622
  • PDF Version: 
    (PDF, 201.23 KB)

Summary: Computerized clinical decision support (CDS) and the use of electronic medical records (EMRs) can improve clinical decisionmaking, adherence with evidence-based guidelines, and quality of care. However, the implementation of CDS into clinical settings is not well understood, and poor integration can hinder its use and minimize its benefits. Common barriers to implementation include poor interface design, usability problems, and failure to accommodate the workflow of a clinical environment.

This project is a field study and controlled simulation analysis on integrating CDS for colorectal cancer screening into outpatient clinical workflow. The team used key informant interviews to identify sitespecific best practices; direct observation of colorectal cancer screening CDS to identify barriers and facilitators to workflow integration; rapid prototyping of design alternatives based on findings from the direct observations; and controlled simulation to test the impact of design on efficiency, usability, and workload. The three study participants - the Regenstrief Institute, the Department of Veterans Affairs (VA), and Partners Healthcare System - use different EMRs but according to a systematic review published in 2006, are all institutions that have improved quality and efficiency by using CDS.

During the first phase, the team conducted site visits to collect qualitative data on factors for effective integration of CDS into clinical workflow in different EMRs. The second phase used measurable attributes - including efficiency, usability, and workload - from the first phase to develop and test alternatives for improved clinical workflow integration in a simulated setting with experienced users.

Project Objectives:

  • Identify key approaches to CDS development for colorectal cancer screening at two VA Medical Center sites and two nationally recognized non-VA sites to obtain effective CDS integration into clinical workflow. (Achieved)
  • Develop and test CDS design alternatives for improved integration into clinical workflow through a controlled simulation study and subsequent implementation. (Achieved)

2011 Activities: The project team completed all work on this project by the end of 2010 and spent most of the first few months of 2011 disseminating the findings. Members of the research team presented a poster, Impact of a Redesign for Colorectal Cancer Screening Computerized Decision Support and a presentation, Investigating Integration of Computerized Decision Support into Workflow at Three Benchmark Institutions at the Veterans Affairs Health Services Research and Development Service 28th Annual National Meeting in February 2011. A manuscript, Redesign of a Computerized Clinical Reminder for Colorectal Cancer Screening: A Human-Computer Interaction Evaluation describing the results was published in the November 2011 volume ofBMC Medical Informatics and Decision Making. Findings from this study directly informed the design of a prototype cancer screening support tool funded by a contract from the Centers for Disease Control and Prevention.

Impact and Findings: EMR and CDS systems differed across the sites. Despite design differences, there were common generalizable barriers, including: 1) lack of coordination among "outside" exam results, primary care, and specialty care; 2) poor data organization and presentation; 3) omission of provider and patient education in the decision support tool; 4) lack of interface flexibility; 5) the need for technological enhancements; 6) unclear role assignments; 7) organizational issues; and 8) disconnect between decision support and quality reporting.

Design enhancements to the VA's existing colorectal cancer screening clinical reminder positively impacted aspects of usability and workflow integration, but not workload. The qualitative analysis revealed broad participant support for the design enhancements and specific suggestions for improving them further. This type of lab-based human-factors evaluation of CDS and other informatics tools is critical for testing design changes prior to implementation.

Overall, the team found that identifying effective strategies in the design, implementation, and integration of CDS into workflow is crucial for effective cognitive support. Despite the use of several different health systems, barriers were quite consistent. Effective design and integration of new technologies requires mindful iteration. There is a need for new CDS prototypes that: 1) improve data organization and presentation; 2) integrate outside results; and 3) provide just-in time education and cognitive support. Designing and testing prototypes using these features may help inform the next generation of cognitive support.

Target Population: Adults, Cancer

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decision making through the use of integrated data and knowledge management.

Business Goal: Implementation and Use

Implementing and Improving the Integration of Decision Support into Outpatient Clinical Workflow - 2010

Summary Highlights

  • Principal Investigator: 
  • Organization: 
  • Contract Number: 
    290-06-0013-3
  • Project Period: 
    September 2007 – March 2011, Including No-Cost Extension
  • AHRQ Funding Amount: 
    $394,622
  • PDF Version: 
    (PDF, 285.94 KB)


Target Population: Adults, Cancer

Summary: Computerized clinical decision support (CDS) and the use of electronic medical records (EMRs) can improve clinical decisions, adherence with evidence-based guidelines, and quality of care. However, the implementation of CDS into clinical settings is not well understood and poor integration can hinder its use and minimize its benefits. Common barriers to implementation include poor interface design, usability problems, and failure to accommodate the workflow of a clinical environment.

This project is a field study and controlled simulation analysis on integrating CDS for colorectal cancer screening into outpatient clinical workflow. The team used key informant interviews on site-specific best practices; direct observation of colorectal cancer screening CDS to identify barriers and facilitators to workflow integration; rapid prototyping of design alternatives based on findings from the direct observations; and controlled simulation to test the impact of design on efficiency, usability, and workload. The three study participants—the Regenstrief Institute, the Department of Veterans Affairs (VA), and Partners Healthcare System—use different EMRs but are all institutions that have improved quality and efficiency by using CDS.

During the first phase, the team conducted site visits to collect qualitative data on factors for effective integration of CDS into clinical workflow in different EMRs. In the second phase, measurable attributes from phase one, including efficiency, usability, and workload were used to develop and test alternatives for improved clinical workflow integration in a simulated setting with experienced users.

Project Objectives:
  • Identify key approaches to CDS development for colorectal cancer screening at two VA Medical Center sites and two nationally recognized non-VA sites to obtain effective CDS integration into clinical workflow. (Achieved)
  • Develop and test CDS design alternatives for improved integration into clinical workflow through a controlled simulation study and subsequent implementation. (Achieved)

2010 Activities: During 2010, the project team continued to analyze data from the first phase of the project and develop manuscripts. Additionally, the team conducted and analyzed data from experiments in the second phase. For this phase, the team developed prototyped design enhancements to the Veterans Health Administration’s (VHA’s) colorectal cancer (CRC) screening clinical reminder to compare with the VHA’s current CRC reminder. These enhancements were based on barriers discovered during the field work in the first phase. In a controlled simulation experiment, 12 primary care providers used prototypes of the current and redesigned CRC screening reminder in a within-subject comparison for four simulated patient encounters. Quantitative measurements were based on a usability survey, workload assessment instrument, and workflow integration survey. They also used ‘think aloud’ techniques during the scenarios and a debriefing session to collect qualitative data.

This work has resulted in numerous dissemination activities, including a poster titled, “Impact of a Redesign for Colorectal Cancer Screening Computerized Decision Support” and a presentation titled, “Investigating Integration of Computerized Decision Support into Workflow at Three Benchmark Institutions”, which were presented for the Veterans Affairs Health Services Research and Development Service 28th Annual National Meeting in February 2011.

Preliminary Impact and Findings: The team found very different forms of EMRs and CDS across the sites. Despite design differences, there were common generalizable barriers. These barriers included: 1) lack of coordination among “outside” exam results, primary care, and specialty care; 2) poor data organization and presentation; 3) omission of provider and patient education in the decision support tool; 4) lack of interface flexibility; 5) the need for technological enhancements; 6) unclear role assignments; 7) organizational issues; and 8) disconnect between decision support and quality reporting.

Design enhancements to the VHA’s existing CRC screening clinical reminder positively impacted aspects of usability and workflow integration, but not workload. The qualitative analysis revealed broad support across participants for the design enhancements with specific suggestions for improving them even further. This type of lab-based human factors evaluation of CDS and other informatics tools is critical for testing design changes prior to implementation.

Overall, the team found that identifying effective strategies in the design, implementation, and integration of CDS into workflow is crucial for effective cognitive support. Despite the use of several different health systems, barriers were quite consistent. Effective design and integration of new technologies requires mindful iteration. New CDS prototypes are needed which: 1) improve data organization and presentation; 2) integrate outside results; and 3) provide just-in time education and cognitive support. Designing and testing prototypes using these features may help inform the next generation of cognitive support.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Synthesis and Dissemination

Integrating Clinical Decision Support Into Workflow - Final Report

Citation:
Doebbeling BN, Saleem J, Haggstrom D, et al. Integrating Clinical Decision Support Into Workflow - Final Report. (Prepared by Indiana University, Regenstrief Institute under Contract No. 290-06-0013-3.) AHRQ Publication No. 11-0076-EF. Rockville, MD: Agency for Healthcare Research and Quality. September 2011. (PDF, 468.03 KB)
Principal Investigator: 
Document Type: 
Research Method: 
Medical Condition: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.