Trial of Aggregate Data Extraction for Maintenance of Certification and Raising Quality

As part of maintenance of certification (MOC), board certified physicians are required to participate in a quality improvement project, including reporting on quality measures before and after an intervention. This study, the “Trial of Aggregate Data Extraction for Maintenance of Certification and Raising Quality,” evaluated a method for family medicine physicians to have their quality measures automatically reported to the American Board of Family Medicine (ABFM) in support of the quality improvement activity. In addition to easing the documentation burden on physicians, the project team hypothesized that access to quality measures would drive physicians to choose a project in an area in which their measures were below average, thus resulting in overall improvement in quality.

The specific aims of this project were as follows:

• Test the capacity for exchange of whole-panel, family physician quality measures from clinical networks. 
• Study whether viewing quality measures and comparison to peers will affect the types of self-assessment modules and quality improvement efforts that are chosen by family physicians. 
• Study whether viewing quality measures and comparison to peers will influence the degree of change in outcomes after quality improvement efforts. 

Physicians from four clinical health systems were randomized, clustered by clinic, to intervention and control conditions. Quality measure data from each system was electronically transmitted to ABFM. When logged into their MOC portal, physicians in the intervention arm were presented with a dashboard of their own measures compared to all study physicians. These physicians thus had an opportunity to choose a quality activity in which their measures lagged behind their colleagues. Physicians in the control arm were shown their quality dashboard only after selecting a certification activity.

Although the duration of the trial was increased, nearly 20 percent of the study physicians did not complete a certification activity, and nearly 25 percent of physicians in the intervention arm never reviewed their quality dashboard. Only 27 percent of improvement activities were completed in an area in which the physician’s quality measures were below average, with no difference between the intervention and control groups. At the end of the trial, there no was difference in quality measure improvement between the two arms.

Reflecting on their findings, the team revised their assumption that physicians review and use quality measures to inform their work. The team concluded that the processes, culture, and assistance needed to support regular quality review and targeted improvement activities may not be in place for many health systems. As a result, emerging value-based payments may not have the expected impact, unless practices and health systems find the resources to support process change for quality improvement reporting.