Evaluating electronic health record data for use in diabetes quality reporting - 2011
Summary: Adoption rates of electronic health record (EHR) systems and pay-for-performance (P4P) programs have been increasing. These two innovations have now merged as the Center for Medicare and Medicaid Services (CMS) has started to permit the use of EHR data for reporting P4P quality measures. While research exists on the challenges of using claims and registry data for P4P programs, there is little guidance on how to use EHR data for quality reporting. Few studies have explored the validity of using EHR data to identify a target population or examined specifically how the use of EHR data can impact quality measures. In addition, there is little information on the accuracy and consistency of EHR data, specifically on how physicians use the EHR data to store information about patient diagnoses.
Despite this lack of information, P4P programs are expanding their use of EHR data, which may be a particular concern given that existing quality measures are often perceived by physicians to not accurately measure quality. Before using this data for quality reporting, it is critical to understand how physicians use the EHR and what motivates their choices.
Ms. Hirsch is evaluating the validity of using EHR data in P4P measures, specifically focusing on quality measures in diabetes, one of the conditions for which CMS permits the use of EHR data. She is evaluating the validity of using different combinations of EHR data elements to identify patients with diabetes by comparing these EHR approaches to a gold-standard manual medical record review and determining how different approaches to using EHR data impact the proportion of patients who meet quality standards. The findings will provide guidance, which is currently unavailable, on how to best leverage existing EHR data to monitor health care quality.
Specific Aims:
- Evaluate the validity of different EHR criteria in identifying primary care patients with diabetes. (Ongoing)
- Determine the impact of utilizing different EHR criteria on existing quality measures for diabetes. (Ongoing)
- Learn where in the EHR physicians document diagnoses of diabetes and identify motivators of documentation behaviors. (Achieved)
2011 Activities: The majority of the work for this project, funded through the Health Services Research Dissertation grant program (R36), was completed in 2011. Activities included requesting, receiving, and cleaning the data extracts for the eight approaches to identifying diabetics using EHR data from the Geisinger Health System. The project team conducted a medical chart review to establish the gold standard of diabetic diagnosis with which to compare the eights sets of criteria. This included analyzing de-identified clinical notes from the EHR. At the end of 2011, data analysis was ongoing. To learn where in the EHR physicians document diagnoses of diabetes and identify motivators of documentation behaviors, 17 semi-structured interviews were conducted with primary care providers. One change from the original application was to include physician assistants and nurse practitioners in addition to physicians. Ms. Hirsch developed a coding scheme and qualitative analysis of the interviews is complete. Two manuscripts from this aim were drafted but are on hold until analyses for the first two aims are complete.
Budget spending was lower than anticipated due to lower costs associated with interviews because of reaching saturation on the qualitative interviews earlier than anticipated. Therefore Ms. Hirsch is using a 3- month no-cost extension (NCE) to continue to develop manuscripts. The NCE was approved through the end of June 2012.
Preliminary Impact and Findings: The most significant finding so far is that organizational factors are a major driver of documentation in the EHR. These organizational factors include pressure from leadership (e.g., encouragement to document information in a particular way), institutional procedures (e.g., automatic generation of diagnoses when providers place specific clinical orders), and existing internal quality programs (e.g., providers are more motivated to document diagnoses if participation in quality initiatives is dependent on documenting the diagnosis in the EHR). The influence of organizational factors on EHR data makes it much harder to come up with standard EHR specifications for performance measurement that can be applied across institutions.
There are some unintended consequences of using EHR data to drive quality management efforts. For example, physicians have expressed concerns about how diagnoses generated by EHR data algorithms might be used (e.g., labeling someone in the EHR that could have insurance implications or a patient finding out about a diagnosis through a patient portal before being told by the physician.)
Target Population: Chronic Care*, Diabetes
Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decision-making through the use of integrated data and knowledge management.
Business Goal: Knowledge Creation
* This target population is one of AHRQ's priority populations.