Massachusetts Quality e-Measure Validation Study

Implementing quality and performance measures has been seen as one way of improving the quality of health care delivery. Nationally standardized performance measurement sets have been endorsed by organizations such as the National Quality Forum (NQF) and the Ambulatory Quality Alliance (AQA). However, efforts to implement these sets have been hampered by the difficulty and expense of collecting data, frequently done manually due to immature electronic systems. With the increase in adoption of electronic health records (EHRs) the potential to be able to collect measurement data with greater ease and accuracy has increased. However, given that clinical data is held in a variety of places — including physician practices, hospitals, emergency departments, laboratories, pharmacies and insurers – robust measurement collection can only be achieved via health information exchanges (HIEs). Little research has been done on the feasibility of measurement collection via an HIE.

This project evaluated the readiness and feasibility of using EHR data for ambulatory clinical performance measurement in a community-wide, multi-payer HIE. The setting for this project was the Massachusetts e-Health Collaborative (MAeHC), which has implemented hundreds of interoperable EHRs in ambulatory practices in three Massachusetts communities, created an HIE, and developed a Quality Data Warehouse (QDW). The QDW was developed in part to enhance the ability for selective retrieval, linkage, and storage of clinical data elements which could be used to calculate clinical quality measures.

The main objectives of this project were to:

• Recruit a cohort of adult ambulatory patients from two communities that are piloting community-wide implementation of structured EHRs to compare a quality measurement method based on structured EHR data to a hybrid method involving a combination of aggregated claims data and medical record review.
• Compare a measurement method based on structured EHR data to a claims-only method based on a novel database that aggregates claims data from commercial health plans and Medicare.

The measures selected for this project came from the AQA ambulatory care measurement set. Of the 26 AQA endorsed quality measures, 14 adult measures were used for implementation into the QDW. The team compared quality measures with two standard measurement methods: the first a “hybrid method” which combined claims data with a medical record review in ambulatory practices; and the second a “claims-only method” which used claims data aggregated across commercial health plans and Medicare.

The project team was unable to complete the planned comparative analysis as had been proposed at the start of the project. A number of barriers were encountered including the largest community not becoming active in the HIE during the timeframe of the project. Other barriers noted included: 1) slow and incomplete implementation of community HIE; 2) technical problems with HIE; 3) legal and regulatory barriers to use of HIE data for evaluation; 4) lack of consistency across institutional review boards and other legal entities reviewing and overseeing the data evaluation protocol; 5) challenge to engaging patients in evaluation of HIE; 6) sampling challenges; 7) incomplete cooperation of community physicians; and 8) low response rates to health plan member survey. The project team felt that knowledge of these barriers will help others as they undertake similar efforts.