Evaluating Electronic Health Record Data for Use in Diabetes Quality Reporting

The Centers for Medicare and Medicaid Services (CMS) has started to permit the use of electronic health record (EHR) data for reporting quality measures for performance measurement. There is little guidance on how to use EHR data for quality reporting. Few studies have explored the validity of using EHR data to identify a target population, or examined specifically how the use of EHR data can impact quality measures. In addition, there is little information on the accuracy and consistency of EHR data, specifically on how physicians use the EHR data to store information about patient diagnoses. Despite this lack of information, pay-for-performance programs are expanding their use of EHR data, which may be a particular concern given that existing quality measures are often perceived by physicians to not accurately measure quality. Before using this data for quality reporting, it is critical to understand how physicians use the EHR and what motivates their choices of where and how they document diagnosis and treatment. 

This project used a mixed-method approach to investigate the validity of using EHR data for diabetes performance measures. The first component consisted of one-on-one telephone interviews with primary care clinicians employed by Geisinger Health System, an integrated health care system in Pennsylvania that began implementing its EHR in outpatient clinics in 1996. The purpose of these key informant interviews was to gain an understanding of how clinicians enter data in the EHR when diagnosing and treating patients with diabetes and what may motivate their documentation behavior such as workflow or billing processes. The second component of the study consisted of EHR data extraction to identify primary care patients with diabetes using eight different EHR-based methods of identification. The validity of the eight methods was evaluated by comparing them to the gold standard of a manual medical record review.

Four themes were identified from the interviews with clinicians. First, clinicians identified two data fields, the problem list diagnosis field and the encounter diagnosis field, as the locations in the EHR where they most frequently document a diagnosis of diabetes. Second, clinicians endorsed the use of a problem list diagnosis for identifying patients with diabetes for quality measures, while they expressed concern that depending on an encounter diagnosis could result in over- or under-identifying patients. Third, organizational factors have an influence over how diagnosis data is entered into the EHR including workflow, internal quality performance programs, and leadership pressure. Fourth, clinicians expressed concerns about the unintended consequences of using EHR data for performance measurement and quality reporting purposes including negative impact on care processes, insurance coverage issues, and unnecessary patient anxiety.

For the second component of the study, the EHR-based methods for identifying patients with diabetes had high specificity (greater than 99 percent) and moderate to high sensitivity (65 to 100 percent). The method of identifying patients with diabetes did not have an impact on the performance measures. However, the EHR criteria used in each of the definitions selectively identified patients who had better quality performance scores.