Closing the Feedback Loop to Improve Diagnostic Quality (Alabama)

Project Final Report (PDF, 154.24 KB) Disclaimer

Project Details - Ended

Project Categories

Summary:

Distinct from the inpatient setting, ambulatory-based clinicians rarely have the opportunity to witness the impact of their treatment in a timely fashion. They therefore rarely learn whether their assessment was accurate and whether the prescribed treatment accomplished the desired effect. This lack of feedback means that incorrect diagnoses may not be detected, medication adverse events may not be recorded, and issues with medication adherence may not be addressed. This project developed an automated system to provide feedback on patient outcomes to clinicians working in ambulatory settings.

The main objectives of this project were to:

  • Develop a system within three different ambulatory electronic health record systems in three different types of ambulatory settings that includes proactive followup of patients’ response to treatment (including medication adherence and adverse events) and feedback to health care providers.
  • Assess the impact of automating the followup and feedback system. Impact will be measured in terms of: 1) diagnostic quality; 2) prevention of adverse events; 3) patient satisfaction with clinical care; and 4) health care costs.
  • Develop and evaluate an automated system for feedback to emergency medicine physicians of the concordance between their initial diagnoses and patients’ final diagnostic outcomes.

There were three data collection phases in the primary study: 1) a telephone call to the patient 1 week after a visit with feedback provided to the physician; 2) contact with the patient via an interactive voice response (IVR) system, also conducted 1 week after a visit with feedback provided to the physician; and 3) a telephone call to the patient 3 weeks after a visit with no physician feedback provided. Measures included problem resolution, medication adherence, patient satisfaction, physician satisfaction, and cost impact. A secondary study conducted in the emergency department (ED) setting looked at the agreement between diagnoses made in the ED versus those given at the time of hospital discharge.

Approximately 15 percent of patients contacted at 1 week by telephone and IVR stated that their presenting issue remained unresolved. The IVR system reached fewer patients than the group who received a telephone call. Of those reached by IVR, a greater number of patients had problems which remained unresolved. Most of the individuals who did not improve had not contacted their providers to let them know their problem had continued.

Physicians who reviewed the provided feedback reported finding it helpful. Patient satisfaction was high for all three phases of the study, with those patients who received followup calls reporting more satisfaction with their overall care than those who did not receive calls. In the ED portion of the study, the diagnoses made in the ED differed from those made at the time of hospital discharge 10 percent of the time. The project team concluded that timely followup is feasible in the ambulatory setting and may catch issues at an earlier stage. The cost for such a system can be offset by an increase in revenue, improvements in the quality of care, and reduced costs due to the avoidance of hospitalizations.

Closing the Feedback Loop to Improve Diagnostic Quality - 2011

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS07-002: Ambulatory Safety and Quality Program: Enabling Quality Through Health Information Technology (EQM)
  • Grant Number: 
    R18 HS 017060
  • Project Period: 
    November 2007 - August 2011
  • AHRQ Funding Amount: 
    $998,509
  • PDF Version: 
    (PDF, 172.18 KB)

Summary: The inpatient setting allows alterative diagnoses to be considered whenever there is a lack of response to therapy or when an adverse event occurs. Determining whether a diagnosis is correct in an outpatient setting may be difficult because patients with inaccurate diagnoses may fail to follow up, get better on their own despite the inaccurate diagnosis, or seek care elsewhere, so that the original provider never learns of the error. Indeed, a "correct" diagnosis may not be discovered until a later date when a biopsy, autopsy, hospital stay, or adverse event occurs and establishes a disparate diagnosis.

This project sought to track outcomes of initial diagnosis and to provide that information to clinicians to give feedback and the opportunity to revise initial diagnoses. The assessment of outcomes was assisted by the involvement of patients. The metric for the quality of the diagnosis was whether the patient's condition resolved in a timely and appropriate manner, or whether - as the result of feedback the initial diagnosis - was modified in a timely manner.

The project developed automated processes for proactive followup and ongoing rapid feedback to physicians in two types of outpatient settings: 1) three ambulatory clinics (the University of Alabama at Birmingham [UAB]-Huntsville Family Practice; United Cerebral Palsy Clinic [UCP]; and the UAB-HIV Clinic); and 2) an emergency department (ED) (Shands-Jacksonville). The ambulatory sites each used different electronic health records (EHRs). UCP used the WorldVistA EHR, UAB used Touchworks EHR, and UAB-HIV used a proprietary EHR. The ED site used the McKesson Horizon Patient Folder and a proprietary ED system that provided a computer-generated paper template that was customized to the patient's chief complaint.

Different interventions were used at each type of site. The clinic site intervention was an interactive voice response (IVR) system that collected followup data for a feedback report to physicians on patient health status and medication adherence. The feedback report used an interface between the EHR and a database that can be integrated with a variety of systems. The ED intervention was an automated followup and feedback report to the ED physicians on the final diagnoses of patients who were admitted to the hospital as compared to their initial ED diagnoses.

Outcome measure included providers' responses to the feedback; satisfaction with the process; its impact on diagnostic and therapeutic quality; response to use of the IVR and ED feedback systems; and use of the feedback by physicians. For the clinic sites, additional assessments included patient satisfaction and impact on health care costs.

Specific Aims:

  • Develop a system within three different ambulatory EHR systems in three different types of ambulatory settings that includes proactive followup of patients' response to treatment (including medication adherence and adverse events) and feedback to health care providers. (Achieved)
  • Assess the impact of automating the followup and feedback system. Impact will be measured in terms of: 1) diagnostic quality; 2) prevention of adverse events; 3) patient satisfaction with clinical care; and 4) health care costs. (Achieved)
  • Develop and evaluate an automated system for feedback to emergency medicine physicians of the concordance between their initial diagnoses and patients' final diagnostic outcomes. (Achieved)

2011 Activities: The 1-year no-cost extension provided the opportunity to complete data collection, analysis, and writing of results. Manuscripts under development during this period included a descriptive paper summarizing the results of the concordance analyses; lessons learned about implementing IVR for ambulatory followup; patient satisfaction results; and a main paper summarizing the whole study and outcomes, including clinical, costs, and physician and patient satisfaction. Presentations on project results were given, including a poster presentation focusing on the patient satisfaction survey at the American Medical Informatics Association spring meeting, and a second presentation on the development of the medication compliance scale presented at the Society for Behavioral Medicine. Dr. Berner also presented a Webinar about the project to researchers at Creighton University in March. As last self-reported in the AHRQ Research Reporting System, project progress was on track and project budget spending was on target. All project activities were completed when the project ended in August 2011.

Impact and Findings: Baseline data showed that 10-to-20 percent of ambulatory patients reported that their problems were not resolved within a week of their acute care visit. Many reported that their problem persisted after 3 weeks. A large proportion of patients did not contact their health care providers when they did not improve as expected. Patient satisfaction with the program was high throughout all phases of the program.

Physicians who viewed the feedback found it helpful. Cost analyses showed that if a followup system was implemented routinely the expense could be offset by increased revenue from return visits, with the potential to improve the quality of care and avert higher costs of hospitalizations. In the secondary study, the overall dissonance rate between ED and discharge diagnoses was approximately 10 percent. Providing feedback to physicians that could address the discordant diagnoses must address the workflow, confidentiality, and time constraints inherent in an ED setting.

Patients and providers appreciate "closing the feedback loop". Patients who receive followup calls are more satisfied with their overall care than those who do not. The project demonstrated that IVR systems are a feasible approach for patient followup in ambulatory settings. Costs for such followup can be offset by increased patient care revenue, and early followup may avert more costly health care expenses and can potentially improve the quality of care.

Target Population: Adults, Cerebral Palsy, HIV/AIDS

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management. 

Business Goal: Synthesis and Dissemination

Closing the Feedback Loop to Improve Diagnostic Quality - 2010

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS07-002: Ambulatory Safety and Quality Program: Enabling Quality Through Health Information Technology (EQM)
  • Grant Number: 
    R18 HS 017060
  • Project Period: 
    November 2007 – August 2011, Including No-Cost Extension
  • AHRQ Funding Amount: 
    $998,509
  • PDF Version: 
    (PDF, 337.22 KB)


Target Population: Adults, Cerebral Palsy, HIV/AIDS

Summary: Determining whether a diagnosis is correct in an outpatient setting may be very difficult. A surrogate measure of diagnostic quality is whether the diagnosis appropriately resolves the reason for the patient’s visit when new complaints or diagnoses arise during the visit. The proposed system focuses on mitigating the harm from an initial diagnosis that does not resolve the patient’s underlying problem. The hypothesis is that harm can be prevented or mitigated by providing rapid feedback to the physician, thereby closing the diagnostic loop.

The project is developing automated processes for proactive followup and ongoing rapid feedback to physicians in two types of outpatient settings: three ambulatory clinics – the University of Alabama at Birmingham (UAB)-Huntsville Family Practice, the UAB-HIV Clinic, and United Cerebral Palsy (UCP) – and one emergency department (ED) setting (Shands-Jacksonville). The ambulatory sites all have different electronic health records (EHRs). The EHRs are the Certification Commission for Health Information Technology (CCHIT)-certified WorldVistA EHR (UCP), the Touchworks EHR (UAB), and a non CCHIT-certified proprietary EHR (UAB-HIV). In the ED study, the systems are the CCHIT-certified McKesson Horizon Patient Folder and a proprietary ED system (Xpress Charts) that provides a computer-generated paper template customized to the patient’s chief complaint.

Different interventions are used at each type of site. The clinic site intervention is an interactive voice response (IVR) system that collects followup data for a feedback report to physicians on patient health status and medication adherence. The feedback report uses an interface between the EHR and a database that can be integrated with a variety of systems. The ED intervention is an automated followup and feedback report to the ED physicians on the final diagnoses of patients who were admitted to the hospital as compared to their initial ED diagnoses.

Providers’ responses to the feedback, their satisfaction with the feedback process, the impact on diagnostic and therapeutic quality, response to use of the IVR and ED feedback systems, and the use of the feedback by physicians will be assessed as outcome measures. For the clinic sites, additional assessments include patient satisfaction and impact on health care costs.

Specific Aims:

  • Develop a system within three different ambulatory EHR systems in three different types of ambulatory settings that includes proactive followup of patients’ response to treatment (including medication adherence and adverse events) and feedback to health care providers. (Achieved)
  • Assess the impact of automating the followup and feedback system. Impact will be measured in terms of: 1) diagnostic quality; 2) prevention of adverse events; 3) patient satisfaction with clinical care; and 4) health care costs. (Ongoing)
  • Develop and evaluate an automated system for feedback to emergency medicine physicians of the concordance between their initial diagnoses and patients’ final diagnostic outcomes. (Ongoing)

2010 Activities: The development of the feedback system in each setting was completed and feedback was provided to physicians in each of the active study sites. The IVR system was developed and implemented to collect feedback data from two sites, the UAB HIV clinic and the Huntsville Family Medicine clinic. The IVR was adapted to use the voice of one of the project interviewers so that the patients would hear a familiar voice. Since the patients in the second clinic did not know their interviewer, the team used the same voice but customized the recordings for the clinic. Using the IVR, they completed data collection from patients in both clinics; the information was then fed back to their physicians.

The process for providing feedback to emergency medicine physicians was begun. It was initially done by having project staff meet directly with the residents. However, despite being well-received, that process did not progress as expected due to challenges with scheduling the sessions and that patient charts were not always available at the meeting locations. The team is working on other strategies to provide feedback in a more feasible manner.

Grantee’s Most Recent Self-Reported Quarterly Status (as of December 2010): The project is progressing well with data collection and manuscript development. Most aims are on track and budget spending is roughly on target.

Preliminary Impact and Findings: Within the two participating ambulatory clinics, a comparison of baseline with one week followup showed that approximately 85 percent are as better by one week as by three weeks. Patient satisfaction surveys showed no significant differences in overall satisfaction between baseline and one week followup. Patients were very positive to the idea of followup and the 46 who reported being called were positive about the actual phone call and had higher overall satisfaction.

Within the ED substudy, the overall rate of dissonance was approximately 10 percent of cases, ranging from 7.7 to 13.8 percent. The ED was the source of the discrepancy in two-thirds of cases, the remainder equally divided between inpatient services and coding errors. There was no evidence for association of dissonance with acuity, as measured by triage class, admitting service, specialty, admission diagnosis, age, race, or gender. There was no evidence for association with ED length of stay, boarding time, or hospital length of stay. There was no evidence of association with resident or attending physician.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Synthesis and Dissemination

Closing the Feedback Loop to Improve Diagnostic Quality - Final Report

Citation:
Berner E. Closing the Feedback Loop to Improve Diagnostic Quality - Final Report. (Prepared by University of Alabama at Birmingham under Grant No. R18 HS017060). Rockville, MD: Agency for Healthcare Research and Quality, 2012. (PDF, 154.24 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
Medical Condition: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.