Using Precision Performance Measurement to Conduct Focused Quality Improvement (Illinois)

Project Final Report (PDF, 195.22 KB) Disclaimer

Using Precision Performance Measurement to Conduct Focused Quality Improvement - 2011

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS07-006: Ambulatory Safety and Quality Program: Improving Quality Through Clinician Use of Health Information Technology (IQHIT)
  • Grant Number: 
    R18 HS 017163
  • Project Period: 
    September 2007 - August 2011
  • AHRQ Funding Amount: 
    $1,199,415
  • PDF Version: 
    (PDF, 201.94 KB)

Summary: Measures that utilize data collected for administrative use, such as billing data, may have inaccuracies at the individual patient level. A quality measure may be recorded as not having been met because a patient was incorrectly considered to be eligible or refused the intervention, or because the appropriate data were not captured. As a result of these limitations, clinicians may reach their quality benchmark targets but still be reported as having fallen short. A health care system that delivers near 100 percent high-quality care for chronic disease care and prevention must rely upon precise measurement methods. Quality measurement needs to be embedded within electronic health record (EHR) systems and become dynamic, accurate, and detailed to support the highest level of care possible for all patients.

This project created systems that allowed clinicians to capture reasons for not providing care as part of point-of-care clinical decision support reminder systems to improve data quality and seamlessly link data to practice-level quality improvement programs and point-of-care interventions. The project used previously-developed quality measurement programs that examine EHR data to measure quality of care for coronary artery disease, heart failure, diabetes, hypertension, and preventive services. This study began at a large academic internal medicine practice and was then implemented in four community practices that use a common EHR.

Exception codes for 18 national quality measures were introduced into the EHR. These measures have been developed by organizations such as the Physicians' Consortium for Performance Improvement at the American Medical Association, or adapted from measures of the National Committee for Quality Assurance. The statistical significance of changes was assessed with time-series analysis. In addition, physicians were repeatedly surveyed on their attitudes toward the interventions. Outcomes of the quality improvement activities were monitored along with the costs of the intervention.

The project consisted of two phases. Phase 1 interventions included point-of-care reminders, linked order sets, point-of-care tools within reminders for documenting exceptions (i.e., patient refusals, inability to afford medications, and contraindications or adverse reactions to recommended interventions), quarterly performance reports, and monthly lists for each physician of their patients who were not prescribed "essential" medications. In addition, there was a patient-focused intervention: if a patient refused a recommended procedure and the physician documented this, the patient was sent information about the benefits of the intervention (e.g., medication or preventive service) and contacted to see if s/he wanted to change his/her decision and receive the intervention. In addition to the interventions described above, Phase 2 included printing a list of unsatisfied quality measures for physicians to review before entering the examination room.

Specific Aims:

  • Integrate simple, standard ways for clinicians to document patient reasons or medical reasons for why quality measures are not met and assess the use of these exception codes, the impact of exception reporting on measured levels of quality, and the impact of using these codes on physician satisfaction and self-reported efficiency. (Achieved)
  • Use the exception codes (patient reasons and medical reasons) that clinicians enter to target three forms of quality improvement, including: 1) peer review of all medical reasons for not adhering to guidelines followed by academic detailing if a clinician enters an unjustified reason for not following guidelines; 2) counseling for patients whose physician enters an exclusion code stating that the patient cannot afford a needed medication, to determine ways of overcoming barriers; and 3) educational outreach to all patients who refuse recommended interventions, including mailing of plain-language health education materials or DVDs. (Achieved)
  • Provide clinicians with highly accurate information on patients' quality deficits immediately prior to their visit as part of routine workflow, and assess whether this intervention increases provision of recommended therapies and tests and documentation of exclusion codes. (Achieved)

2011 Activities: The research team finished collecting and analyzing data by the end of the 1-year no-cost extension. As last self-reported in the AHRQ Research Reporting System, project progress and activities were completely on track and the project budget was somewhat underspent, approximately 5 to 20 percent. The project was completed in August 2011.

Impact and Findings: For Phase 1, during the year before the start of the intervention, performance improved significantly for eight measures, did not change for six, and declined for one. Temporal trends could not be calculated for cervical cancer screening because undated exceptions were recorded during the pre-intervention period. During the year after the start of the intervention, performance improved significantly for 14 measures, improved non-significantly for another (hemoglobin A1c control), and declined for one. During the intervention year, the rate of improvement in performance was significantly greater for nine measures and of borderline significance for another. Another four measures improved during the post-intervention period, but the rates of improvement were similar to the pre-intervention period. The rate of improvement in performance for osteoporosis screening was lower during the intervention year than the pre-intervention year. The absolute rate of screening mammography declined, which was attributed to a shortage of trained radiologists and prolonged waiting times at the institution. The improvements in performance during the intervention year were due to a combination of more patients satisfying the measures and documentation of exceptions.

For Phase 2, the addition of paper reminders to the interventions in Phase 1 did not have a marginal benefit overall, and it did not improve performance for the physicians with the worst performance at the end of Phase 1. Performance improved significantly for eight of the 16 measures during Phase 2. Performance had improved significantly during Phase 1 for all of these eight measures. Performance of screening mammography declined significantly during Phase 2; this was already declining in Phase 1, as described above. Performance decreased for two other measures during Phase 2: 1) prescription of anticoagulants for patients with atrial fibrillation and heart failure; and 2) nephropathy screening or management for patients with diabetes. Both of these had previously shown an improvement in performance during Phase 1. Performance did not change during Phase 2 for prescribing antiplatelet drugs for patients with coronary artery disease; performance had increased during Phase 1 and remained stable at a very high level (approximately 95 percent). Glycemic control (hemoglobin A1c < 8 mg/dl) did not change throughout the study.

Target Population: Adults, Chronic Care*, Diabetes, Heart Disease, Hypertension

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Implementation and Use

* This target population is one of AHRQ's priority populations.

Using Precision Performance Measurement to Conduct Focused Quality Improvement - 2010

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS07-006: Ambulatory Safety and Quality Program: Improving Quality Through Clinician Use of Health Information Technology (IQHIT)
  • Grant Number: 
    R18 HS 017163
  • Project Period: 
    September 2007 – August 2011, Including No-Cost Extension
  • AHRQ Funding Amount: 
    $1,199,415
  • PDF Version: 
    (PDF, 359.25 KB)


Target Population: Adults, Chronic Care*, Diabetes, Heart Disease, Hypertension

Summary: Measures that utilize data collected for administrative use, such as billing data, inevitably have inaccuracies at the individual patient level. A quality measure may be recorded as not having been met because a patient was incorrectly considered to be eligible or refused the intervention, or because the appropriate data were not captured. As a result of these limitations, clinicians may in truth reach their quality benchmark targets, but automatic reporting of measures pulled from administrative data fail to accurately reflect this. Imprecise measurement methods can never be the foundation for a health care system that delivers near 100 percent high-quality care for chronic disease care and prevention. Quality measurement needs to be embedded within electronic health record (EHR) systems and become dynamic, accurate, and detailed to support the highest level of care possible for all patients.

This project creates systems that allow clinicians to capture reasons for not providing care as part of point-of-care clinical decision support reminder systems, improve data quality, and seamlessly link data to practice-level quality improvement programs and point-of-care interventions. The project uses previously developed quality measurement programs that examine EHR data to measure quality of care for coronary artery disease, heart failure, diabetes, hypertension, and preventive services. This study began at a large academic internal medicine practice and is now being implemented in four community practices that use the same Certification Commission for Health Information Technology-certified EHR, produced by Epic.

Exception codes are being introduced into the EHR for 18 national quality measures. Data are extracted from the EHR every month to assess changes in the primary outcome: the proportion of eligible patients who do not satisfy a measure and do not have any exclusion criteria documented. The statistical significance of changes will be assessed with time series analysis. In addition, physicians will be repeatedly surveyed on their attitudes toward the interventions described in the aims listed below. Outcomes of the quality improvement activities will be monitored along with the costs of the intervention. This study will produce computerized tools and educational materials that can be provided to more than 1,000 sites that use the Epic EHR ambulatory product.

Specific Aims:
  • Integrate simple, standard ways for clinicians to document patient reasons or medical reasons for why quality measures are not met and assess the use of these exception codes, the impact of exception reporting on measured levels of quality, and the impact of using these codes on physician satisfaction and self-reported efficiency. (Ongoing)
  • Use the exception codes (patient reasons and medical reasons) that clinicians enter to target three forms of quality improvement, including: 1) peer review of all medical reasons for not adhering to guidelines followed by academic detailing if a clinician enters an unjustified reason for not following guidelines; 2) counseling for patients whose physician enters an exclusion code stating that the patient cannot afford a needed medication to determine ways of overcoming barriers; and 3) educational outreach to all patients who refuse recommended interventions, including mailing of plain-language health education materials or DVDs. (Ongoing)
  • Provide clinicians with highly accurate information on patients’ quality deficits immediately prior to their visit as part of routine workflow, and assess whether this intervention increases provision of recommended therapies and tests and documentation of exclusion codes. (Ongoing)

2010 Activities: Implementation at the Northshore site significantly progressed. The site was able to implement the clinical decision support and reminder tools in the EHR for select conditions, and generated individual physician and group-level quality reports that included the data on entered exceptions. Data has been generated for the time series studies to analyze changes in quality of care over the course of the study as measured by increases in patients receiving the service, documentation of exceptions, or a combination of both. Further preparation of the data for the time series studies continued beyond the end of the year. Initial reviews of the validity of the entered medical exceptions have been completed which have shown a high level of validity for entered exceptions, only slightly below that seen at the Northwestern site.

Analyses of the effects of the pre-encounter quality deficit reminder system were completed. Although quality of care continued to improve during this second year of the intervention, the pre-encounter notification system did not appear to engage those physicians who were not frequently using the electronic clinical decision support tools. However, the physician survey suggested that doctors liked it, so it has been continued. The study team began to write up these results.

Grantee’s Most Recent Self-Reported Quarterly Status (as of December 2010): The project’s progress is reported as completely on track and is meeting 100 percent of its milestones on time. Project spending is roughly on target.

Preliminary Impact and Findings: For the first aim, the primary outcome of ten measures significantly improved more rapidly the year after implementation than during the prior year. For four other measures, quality improved, but the rate of improvement did not differ significantly from the year prior to the intervention. One measure improved at a significantly slower rate, and the performance of mammography declined due to new barriers to access at the study site. Improvements resulted from increases in patients receiving the service, documentation of exceptions, or a combination of both. By the end of the first year, for five drug prescribing measures, over half of physicians achieved 100 percent performance.

For the second aim, 6.5 percent of the quality reviews identified an issue requiring feedback from an investigator to a clinician, who then entered a medical exception. In patient outreach, the majority of patients did not want to talk about their refusal. Of all patients, 13.5 percent eventually completed a test or took a medication they originally declined.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Implementation and Use

*AHRQ Priority Population.

Project Details - Ended

Project Categories

Summary:

Quality measurement techniques have increased in sophistication during the past few decades and now allow for meaningful comparisons between health care facilities or health plans. However, as currently practiced, these methods cannot be used to raise health care quality to the highest possible level because measures that depend on data collected for administrative purposes inevitably have measurement inaccuracies at the individual patient level. Patients may incorrectly be considered eligible for a measure; appear to fail a quality measure they have met because data satisfying the measure was not captured; or have reasons the measure was not appropriate for them, such as exclusion criteria that the measurement system failed to detect.

The overall goal of this study was to determine whether a health information technology-enabled quality improvement strategy could improve performance on a set of 18 measures of quality of care for four chronic conditions and five preventive services. The specific aims were to:

  • Create simple, standard ways for clinicians to document patient reasons or medical reasons for why quality measures are not met.
  • Use the exception codes that clinicians enter (i.e., patient reasons and medical reasons for not providing a recommended therapy or preventive service) to target three strategies for quality improvement:
    • Peer review of all medical reasons for not adhering to guidelines followed by academic detailing if a clinician enters an unjustified reason for not following guidelines.
    • Counseling for all patients whose physician enters an exception code stating that the patient cannot afford a needed medication to determine ways of overcoming barriers.
    • Educational outreach to all patients who refuse recommended interventions, including mailing of plain-language health education materials or DVDs.
  • Provide clinicians with highly accurate information on patients’ quality deficits immediately prior to each patient’s visit as part of routine work flow.

The study took place at the Northwestern Medical Faculty Foundation’s General Internal Medicine clinic. Exception codes for 18 national quality measures for four chronic conditions and five preventive services were introduced into the electronic health record. These measures had been developed by organizations such as the Physicians' Consortium for Performance Improvement at the American Medical Association or adapted from measures of the National Committee for Quality Assurance. Two measures were not implemented in the study due to technical limitations: blood pressure control in patients with and without diabetes. The statistical significance of changes was assessed with time-series analysis. In addition, physicians were repeatedly surveyed on their attitudes toward the interventions. Outcomes of the quality improvement activities were monitored as were the costs of the intervention.

During the first year of the intervention, performance improved significantly for 14 measures. For nine measures, the primary outcome improved more rapidly during the intervention year than during the prior year. The improvements in performance during the intervention phase were due to a combination of more patients satisfying the measures and physician documentation of exceptions. The project team found that the medical exceptions documented were almost always valid. For patients that refused recommended services, outreach, such as mailed educational materials and care manager calls to identify and resolve any barriers to obtaining the service, was not effective. For physicians whose overall performance lagged, the paper reminders provided to physicians to review prior to entering the examination room was also not effective.

Using Precision Performance Measurement to Conduct Focused Quality Improvement - Final Report

Citation:
Baker DW. Using Precision Performance Measurement to Conduct Focused Quality Improvement - Final Report. (Prepared by Northwestern University under Grant No. R18 HS017163). Rockville, MD: Agency for Healthcare Research and Quality, 2012. (PDF, 195.22 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.