Bringing Measurement to the Point of Care (New York)

Project Final Report (PDF, 172.96 KB) Disclaimer

Bringing Measurement to the Point of Care - 2010

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    RFA: HS07-002: Ambulatory Safety and Quality Program: Enabling Quality Measurement Through Health Information Technology (EQM)
  • Grant Number: 
    R18 HS 017059
  • Project Period: 
    September 2007 – September 2010, Including No-Cost Extension
  • AHRQ Funding Amount: 
    $694,961
  • PDF Version: 
    (PDF, 352.44 KB)


Target Population: Adults, Inner City*, Low SES/Low Income*, Medicaid, Medically Underserved, Racial or Ethnic Minorities*: African American, Hispanic/Latino, Safety Net, Uninsured

Summary: The overall goal of this project is to implement meaningful measurements of the quality of care that focus on public health priority issues, disadvantaged populations, and small office practices. The New York City Department of Health and Mental Hygiene (DOHMH) Take Care New York initiative has articulated 10 priority public health issues that require coordinated action between health care providers, patients, community organizations, and government agencies. The DOHMH Primary Care Information Project (PCIP) uses health information technology for population-wide measurement and improvement of clinical care in these 10 domains, particularly among disadvantaged populations. Over 1,000 medical providers have implemented electronic health records (EHRs) with enhanced preventive care functionality. Ambulatory Certification Commission for Health Information Technology-certified EHR products include Epic, NextGen, and eClinicalWorks. Clinical partners include New York City’s Federally Qualified Health Centers, several hospital outpatient departments, and hundreds of primary care providers in small office settings. A set of 38 clinical quality measures designed to address priority public health issues has been developed, and automated reporting of these measures internally and to the DOHMH will be coordinated with the EHR vendors.

Successful EHR-enabled quality measurement requires that physicians document relevant information at the point of care. The research team developed a simple and intuitive clinical decision support system (CDSS) with eClinicalWorks, suitable for small office practices, that integrates quality measurement and clinical decision support at the point of care. The CDSS displays a dashboard of quality indicators as part of the patient’s record, showing the patient’s measurement cohorts and whether their care complies with screening and treatment recommendations. Integrated decision support tools enable providers to take appropriate action to bring the measure into compliance or remove the patient from the measurement cohort due to valid exclusions or contraindications.

Limited to small physician owned practices, the project uses a randomized controlled trial to assess the impact of pay-for-quality incentives on quality measurement and improvement across four of the quality measurement areas. A case-control study was conducted to assess the impact of CDSS on quality measurement. The project also uses a pre- versus post- EHR survey to measure the impact of EHR adoption on provider attitudes and engagement with quality measurement and incentivized care.

Specific Aims:

  • Validate a set of automated clinical quality measures that addresses priority public health issues. (Achieved)
  • Characterize provider attitudes and measure provider satisfaction with performance indicators. (Achieved)
  • Design a simple and intuitive point-of-care quality measurement and decision support user interface (quality dashboard). (Achieved)
  • Conduct a randomized clinical trial to determine the impact of this quality dashboard on the accuracy of, and provider satisfaction with, EHR-derived quality measures. (Achieved)
  • Disseminate findings through the National Quality Forum’s Standardizing Ambulatory Care Performance Measures project through the EHR vendors’ participation in this project and through reviewed publications. (Achieved)

2010 Activities: The PCIP EHR development team, in collaboration with eClinicalWorks, completed validation of the automated calculation of selected quality measures. The research team worked with the EHR vendor to test and verify the automated calculation of measures. A separate team compared the monthly data collected from the quality measures against other data sources (e.g., encounter data, self-reported practice and provider characteristics, and potential rise of symptoms related to infectious disease—syndromic surveillance) available within the health department.

The grant team completed development of the provider survey, assessing provider attitudes and satisfaction with the performance indicators. The survey focused specifically on clinical decision support at the point of care, whether any of the features were used, the functionality of the features, and opinions about the features. Survey findings were supported by qualitative data which provided insights into how the technology was and was not working well.

Recruited practices were enrolled in a privately-funded pilot recognition and pay-for-quality program, Health eHearts, because they had adopted an EHR prior to 2009 and were focusing on quality measurement through the incentive program. Of the 84 practices that participated in Health eHearts, 56 practices, representing 154 providers, agreed to electronic record review and abstraction. Interviews were conducted, and two sets of provider surveys were administered to the providers that adopted an EHR through the PCIP program.

Analysis of the manual chart review data was completed and a regression model was used to assess whether practices experienced more rapid increases post-CDSS implementation. Charts were reviewed for three time periods: 1) pre-EHR implementation (likely to be paper documentation), 2) the period from EHR implementation until the reminder CDSS system was implemented (approximately 6 to 18 months), and 3) post-implementation of CDSS system (a minimum of 6 months later). Each patient record was reviewed for documentation to identify whether the patient was eligible for inclusion in the denominator and, if the patient was eligible, whether he or she met the numerator criteria.

Grantee’s Most Recent Self-Reported Quarterly Status (as of December 2010): The grant ended in September 2010 with all major aims achieved. The budget was somewhat underspent.

Impact and Findings: Across 56 practices and reviews of over 6,100 patient records, most practices did not have prior quality measurement experience or engage in quality improvement activities prior to adoption of an EHR. Following implementation of the CDSS and additional technical support from PCIP, practices increased their rates of clinical preventive services by at least five percentage points across six of ten selected quality measures. Providers found some documentation tasks within the EHR to be relatively straightforward, while other data entry processes posed challenges with downstream effects for quality measurement and reporting. Without manual review of the electronic records, many of the practices would underreport their performance when using automated EHR-derived quality reporting.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Implementation and Use

*AHRQ Priority Population.

Project Details - Ended

Project Categories

Summary:

This project assessed the impact and challenges of introducing health information technology tools to assist small primary care practices in measuring quality of care. In collaboration with an electronic health record (EHR) vendor, the Primary Care Information Project (PCIP) deployed new functionality that allowed clinicians to view 'real time' measures of the delivery of clinical preventive services. In addition, the EHR added patient specific alerts and 'one click' options to act on potentially missed opportunities for followup care or recommended preventive services. Providers using these systems were given substantial technical support and training to use these tools.

The primary objectives of this project were to:

  1. Design a simple and intuitive point of care quality measurement and decision support user interface.
  2. Validate a set of automated clinical quality measures that address priority public health issues.
  3. Characterize provider attitudes and measure provider satisfaction with performance indicators.
  4. Determine the impact of the quality dashboard on the accuracy of, and provider satisfaction with, EHR derived quality measures.
  5. Disseminate findings though national measurement consensus organizations (e.g., National Quality Forum's "Standardizing Ambulatory Care Performance Measures" project), through EHR vendors participating in this project, and through peer reviewed publications.

The study found that across the 56 practices involved, most did not have prior quality measurement experience or engage in quality improvement activities prior to the adoption of the EHR. Though most quality measures selected for this study showed post implementation improvement, a review of over 6,100 patient records showed that automated EHR-derived quality reports underreported many of the practices' performance. Providers found some documentation tasks within the EHR to be straight forward, while other data entry processes posed challenges that affect subsequent quality measurement and reporting.

Bringing Quality Measurement to the Point of Care - Final Report

Citation:
Wu, W. Bringing Quality Measurement to the Point of Care - Final Report. (Prepared by New York City Department of Health and Mental Hygiene under Grant No. R18 HS017059). Rockville, MD: Agency for Healthcare Research and Quality, 2011. (PDF, 172.96 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 

Using Data for Measuring Clinical Quality: Provider Attitudes and Experience

This is an interview guide designed to be conducted with physicians in an ambulatory setting. The tool includes questions to assess attitudes of users of electronic health records.

Year of Survey: 
Created prior to 2008
Survey Link: 
Using Data for Measuring Clinical Quality: Provider Attitudes and Experience (PDF, 104.18 KB)
Document Type: 
Research Method: 
Population: 
Copyright Status: 
Permission has been obtained from the survey developers for unrestricted use of this survey; it may be modified or used as is without additional permission from the authors.

Primary Care Information Project (PCIP) Evaluation: Patient Survey

This is a questionnaire designed to be completed by patients in an ambulatory setting. The tool includes questions to assess user's perceptions of electronic health records.

Year of Survey: 
2005
Survey Link: 
Primary Care Information Project (PCIP) Evaluation: Patient Survey (PDF, 247.84 KB) (Persons using assistive technology may not be able to fully access information in this report. For assistance, please contact Corey Mackison)
Population: 
Copyright Status: 
Permission has been obtained from the survey developers for unrestricted use of this survey; it may be modified or used as is without additional permission from the authors.

Primary Care Information Project (PCIP) Evaluation: Provider Survey

This is a questionnaire designed to be completed by physicians in an ambulatory setting. The tool includes questions to assess user's perceptions of electronic health records.

Year of Survey: 
Created prior to 2010
Survey Link: 
Primary Care Information Project (PCIP) Evaluation: Provider Survey (PDF, 96.4 KB) (Persons using assistive technology may not be able to fully access information in this report. For assistance, please contact Corey Mackison)
Population: 
Copyright Status: 
Permission has been obtained from the survey developers for unrestricted use of this survey; it may be modified or used as is without additional permission from the authors.

Primary Care Information Project (PCIP) Post-Electronic Health Record Implementation: Survey of Providers

This is a questionnaire designed to be completed by physicians in an ambulatory setting. The tool includes questions to assess user's satisfaction of electronic health records.

Year of Survey: 
Created prior to 2010
Survey Link: 
Primary Care Information Project (PCIP) Post-Electronic Health Record Implementation: Survey of Providers (PDF, 146.97 KB) (Persons using assistive technology may not be able to fully access information in this report. For assistance, please contact Corey Mackison)
Population: 
Copyright Status: 
Permission has been obtained from the survey developers for unrestricted use of this survey; it may be modified or used as is without additional permission from the authors.

Health eHearts Physician Survey

This is a questionnaire designed to be completed by nurses and physicians in an ambulatory setting. The tool includes questions to assess user's perceptions of electronic health records.

Year of Survey: 
Created prior to 2009
Survey Link: 
Health eHearts Physician Survey (PDF, 170.31 KB) (Persons using assistive technology may not be able to fully access information in this report. For assistance, please contact Corey Mackison)
Population: 
Copyright Status: 
Permission has been obtained from the survey developers for unrestricted use of this survey; it may be modified or used as is without additional permission from the authors.
This project does not have any related story.