Bringing High Performing Systems to Small Practices (New York)

Project Final Report (PDF, 253.56 KB) Disclaimer

Project Details - Ended

Project Categories

Summary:

Despite the evidence base for their role in reducing morbidity and mortality, delivery of clinical preventive services (CPS) has stagnated in the adult primary care setting. In particular, small, independent practices are challenged by the absence of integrated information systems, incorporation of timely and actionable information at the point of care, adequate reimbursement for CPS, shared resources to support technical needs, and quality improvement coaching that is often available in larger integrated care delivery systems. Attempts to boost wide adoption of health information technology (IT) in small practices have been challenging due to costs and lack of technical expertise and support. However, a majority of primary care visits occur in small practices, which highlights the need to understand which systems and resources can be implemented to improve the quality of health care.

This study assessed the effects of supportive electronic health record (EHR) implementation, clinical decision support (CDS) systems, and pay-for-quality programs on the performance of cardiovascular health clinical quality measures (CQM)—aspirin therapy, blood pressure control, cholesterol control, and smoking cessation intervention (ABCS). The project targeted small New York City community providers that had joined the Primary Care Information Project (PCIP), a public health bureau dedicated to integrating health information systems to improve population health. This project also assessed the impact of the interventions on improving the delivery of CPS for the ABCS.

The specific aims of this project were to:

  • Determine whether practices that participated in the PCIP program experienced a more rapid rate of improvement on their quality measures than practices that did not participate.
  • Determine if PCIP-participating practices are atypical in comparison to other small independent practices in New York City.
  • Assess the attributable impact of each intervention: adoption of EHR, CDS, and pilot pay-for quality program.

A mixed-methods approach was utilized across the study aims. Independently owned primary care practices with less than 10 physician staff were recruited for the 2-year pay-for-quality pilot, called Health eHearts. Two cohorts were established from participants of this pilot: early adopters (80) that had implemented the EHR prior to January 2009, and later adopters (60) that had implemented their EHR prior to 2010 but after 2009. All participants adopted the same EHR software.

Practice participants received support and training on using the EHR for tracking and documenting CQM. Practices received quarterly reports on their performance on the ABCS. Half of the practices were randomized to receive incentive payments for achieving patient goals, control of blood pressure, control of cholesterol, or delivering cardiovascular preventive services such as aspirin therapy and smoking cessation intervention.

Early adopters of EHR improved performance on seven of nine CQM in comparison to practices using paper systems, which improved only three of the CQMs. The effect of the CDS tool was mixed. Practice exposure to CDS was not associated with improved performance on ABCS or non-ABCS measures. Receipt of technical assistance was associated with higher performance on quality measures. Financial incentives resulted in modest improvements in cardiovascular processes and outcomes. Contrary to the hypothesis that provider comfort with computers and attitudes (both positive and negative) prior to adoption would predict measures of EHR use after implementation, no significant relationship between attitudes prior to implementation and EHR use were observed. Practices that received financial incentives were more likely to utilize CDS, technical assistance, and review quality reports.

The project team concluded that policymakers who want to accelerate or maximize investments in health IT infrastructure may need to consider payment models coupled with technical support resources to accelerate improvements in areas of preventive care that are underutilized or need improvement.

Bringing High Performing Systems to Small Practices - 2012

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-270: Utilizing Health Information Technology to Improve Health Care Quality Grant (R18)
  • Grant Number: 
    R18 HS 018275
  • Project Period: 
    December 2009 – July 2013
  • AHRQ Funding Amount: 
    $1,199,853
  • PDF Version: 
    (PDF, 296.46 KB)

Summary: To date, there is limited evidence about the ability of small community health care providers to improve quality of care through the use of electronic health records (EHRs), and limited data about financial incentives or technical assistance for quality improvement’s impact on small providers. Investments in health information technology (IT) are being made to improve quality of care, and though there is evidence of improved quality in integrated delivery systems, such as the Kaiser Permanente system, there is less evidence about the effectiveness of health IT on patient outcomes in nonintegrated health systems.

This study is assessing the effects that supportive EHR implementation, clinical decision support (CDS) systems, and pay-for-quality improvements have on small community providers’ clinical preventive services, particularly for cardiovascular health. The New York City Primary Care Information Project (PCIP) is comparing the implementation of EHRs at 80 small ambulatory primary care practices that are early adopters of EHRs and part of an integrated delivery system throughout New York City, to 60 similar
practices in the area that are late adopters of EHRs. The project targets EHR implementation throughout New York City and focuses on some of the poorest neighborhoods.

The study is evaluating the impact of an EHR implemented with the support of technical assistance and added tools, including integrated registry systems and CDS, on improvements in quality-of-care as compared to practices that did not have the aforementioned support. The primary goal has been to determine whether practices that have supportive EHR implementation provide higher-quality care and experience a rapid rate of improvement of their quality measures. A secondary goal has been to determine if any characteristics indicate that supported EHR practices are atypical or have consistently different characteristics than other small independent practices. For the secondary goal, the analysis was conducted by comparing practices that were early adopters of an EHR to those that were late adopters. At a more nuanced level, the research is assessing the attributable impact of various interventions on changes in four areas of preventive services on cardiovascular health (aspirin therapy, blood pressure control, cholesterol control, and smoking cessation intervention) at small practices that provide adult primary care. Results will guide future program design and policies for supporting resources to improve cardiovascular health.

Specific Aims:

  • Determine whether practices that participated in the PCIP program experienced a more rapid rate of improvement on their quality measures than practices that did not participate. (Achieved)
  • Determine if PCIP-participating practices are atypical in comparison to other small independent practices in New York City. (Achieved)
  • Assess the attributable impact of each intervention: adoption of EHR, CDS, and pilot pay-forquality program. (Ongoing)

2012 Activities: The project team completed chart reviews in practices to understand the gains in quality from the use of EHRs. The chart reviews of paper records provided historical data on quality performance prior to the implementation of the EHRs. After the EHRs were implemented, a manual review of electronic records was conducted for comparison. The practices were separated into two cohorts based on whether they were early or late adopters of an EHR. The post-EHR quality performance data was compared to pre-EHR chart data from the late adopters. This cross sectional analysis provided further insight into performance trends pre- and post-EHR adoption, in addition to the analysis of pre- versus post-data from the early adopters.

The team continued an analysis of survey data describing providers’ experiences with quality measurement, reporting, and incentives, as well as a survey of general provider characteristics. A factor analysis of physician motivation for using the EHR was conducted to further understand physician attitude about experience using EHR. The team also linked the survey data on providers’ attitudes to their actions and use of the EHR, such as computerized provider order entry and structured fields.

For the third aim, three different analyses were conducted to understand the attributable impact of various interventions on quality outcomes. The first analysis was the assessment of the impact of graduated financial incentives to meet quality performance targets for cardiovascular patients. The graduated payments were provided for patients who were more difficult to treat because of socio-economic status or chronic conditions in addition to cardiovascular disease. The impact of financial incentives was assessed by comparing the performance improvement of indicators targeted by financial incentive and the performance improvement on indicators not targeted by financial incentives. Multi-regression models adjusting for practice size and characteristics were used to compare pre- and post-performance on a set of quality measures depending on whether practices were randomized to receive incentives.

The second analysis was of the impact of patient-centered medical home (PCMH) recognition on trends in cardiovascular quality measures. Quality-of-care trends for 267 practices were generated and compared by whether practices had recognition as a PCMH; a total of 64 practices had recognition by December 2012. Quality measures trended included: aspirin therapy for patients with ischemic vascular disease and/or diabetes; blood pressure control for patients with hypertension; body mass index recorded for patient 18 years and older; hemoglobin A1c testing for patients with diabetes; smoking status recorded; and smoking cessation intervention for smokers. The third analysis was trends observed pre- and post-EHR and CDS functionality implementation.

An analysis of the effect of each of the individual EHR adoption interventions on 39 quality measures related to chronic disease prevention and management was conducted in partnership with Weil Cornell Medical College. Predictor variables evaluated included time live on the EHR, use of CDS, quality feedback, financial incentives, and technical assistance.

The project team is using a 1-year no-cost extension to complete the analysis and manuscript development of results. As last self-reported in the AHRQ Research Reporting System, project progress is completely on track and project budget spending is on target.

Preliminary Impact and Findings: The preliminary analysis of technical assistance was associated with improved performance on several clinical process measures, with some evidence of a dose response between number of technical assistance visits and degree of improvement. Providers with five visits improved 3.9 percentage points more than those who had received no visits over the study period, and providers who received eight visits improved by 5.6 more percentage points than those who had received
none. In comparing across interventions and their effect on delivery of preventive care on cardiovascular measures, practice exposure to EHRs was not associated with improved quality performance on the targeted measures, and weakly associated with untargeted measures. Similarly, technical assistance was not associated with quality performance on the targeted measures, but was associated with improved quality of care on the untargeted measures for exposure to technical assistance. Exposure to CDS was not associated with performance on either targeted or untargeted measures. However, while financial incentives for quality measures were significantly associated with improved quality of care for the targeted measures, financial incentives were associated with worse performance on the untargeted measures. Preliminary results showed no association between comfort with computers and attitudes toward EHR implementation prior to implementation and measures of EHR use 1 year later. These results hold promise that lack of prior experience and negative attitudes may not impair future meaningful use of health IT.

Target Population: Adults, Inner City*, Low-SES/Low Income*, Medicaid, Medically Underserved, Safety Net

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve
health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

* This target population is one of AHRQ’s priority populations.

Bringing High Performing Systems to Small Practices - 2011

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-270: Utilizing Health Information Technology to Improve Health Care Quality Grant (R18)
  • Grant Number: 
    R18 HS 018275
  • Project Period: 
    December 2009 - November 2012
  • AHRQ Funding Amount: 
    $1,199,853
  • PDF Version: 
    (PDF, 218.21 KB)

Summary: To date, there is limited evidence on the ability of small community health care providers to improve quality of care through the use of electronic health records (EHRs), and limited data on the impact of financial incentives for quality improvement on small providers. Investments in health information technology (IT) are being made to improve quality of care and while there is evidence of improved quality in integrated delivery systems, such as the Kaiser Permanente system, there is less evidence of the effectiveness of health IT on patient outcomes in nonintegrated health systems.

This study provides information on the effects that supportive EHR implementation, clinical decision support (CDS) systems, and pay-for-quality improvements have on small community providers' cardiovascular health outcomes. The New York City Primary Care Information Project (PCIP) is comparing the implementation of EHRs at 60 small ambulatory primary care practices that are early adopters of EHRs and part of an integrated delivery system throughout New York City to 60 similar practices in the area that are late adopters of EHRs. The project targets EHR implementation throughout New York City and focuses on some of the poorest neighborhoods.

The study will evaluate the impact of an EHR implemented with the support from technical assistance and added tools, including integrated registry systems and CDS, on improvements in quality of care as compared to practices that do not have an EHR or the aforementioned support. The primary goal is to determine whether practices that have supportive EHR implementation provide higher-quality care and experience a rapid rate of improvement of their quality measures. A secondary goal is to determine what characteristics, if any, indicate that supported EHR practices are atypical or have any consistently different characteristics as compared to other small independent practices. At a more nuanced level, the research will assess the attributable impact of various interventions on changes in four cardiovascular health outcomes at small practices that provide adult primary care. This will provide specific information on the value of various types of support on the rate of improvement on cardiovascular quality measures.

Specific Aims:

  • Determine whether practices that participated in the PCIP program experienced a more rapid rate of improvement on their quality measures than practices that did not participate. (Ongoing)
  • Determine if PCIP-participating practices are atypical in comparison to other small independent practices in New York City. (Ongoing)
  • Assess the attributable impact of each intervention: adoption of EHR, CDS, and pilot pay-for-quality program. (Upcoming)

2011 Activities: The project team began to analyze the data for examining the effect of each successive stage of health IT implementation on higher-quality performance. This included analysis of a baseline survey describing providers' experiences with quality measurement, reporting, and incentives, as well as a survey of general provider characteristics. The project team continued analysis of baseline survey data that provides information on the characteristics of the practices that are early versus late adopters. EHR adoption among small clinics in New York has moved rapidly since the writing of the grant application, and there are fewer practices that have not begun EHR implementation. As a result, the project shifted the definition of the control practices from non-EHR adopters to a subset of practices that are late adopters of the EHR. For the later adopters, 58 practices representing a total of 134 providers were recruited. Similar to the early adopters, the majority of practices that were recruited as the late adoption group are solo or two-provider practices. The project defines 'early adopters' as those that adopted an EHR prior to January 2009. 'Late adopters' are those that adopted between January 2009 and March 2010.

At 6 months into the implementation of their EHR, each practice was asked to complete a followup survey to provide contextual information on the components of the EHR they were using, and their thoughts on Meaningful Use and other topics. Through the review of clinical outcomes data, the project team is beginning to measure the impact of each successive stage of IT integration. Clinical data are being gathered through chart review and where applicable, electronically. The team developed a form, database, and instruction set to collect the clinical data elements from paper charts that can be used to calculate the same quality measures as those calculated through the EHR. The other metric under review is the relationship between IT implementation and medical-home certification from the National Committee for Quality Assurance.

As last self-reported in the AHRQ Research Reporting System, project progress is mostly on track and the project budget funds are somewhat underspent due to delays in contracting.

Preliminary Impact and Findings: Preliminary analysis was conducted with the early cohort of practices to understand trends in quality measurement before and after EHR adoption, as well as 6-months after use of EHR. Within a cohort of 36 practices, 3,120 patient records were manually reviewed in two time periods prior to EHR adoption, a few months after EHR adoption, and 6-months after EHR adoption. Trends were calculated for the following quality-of-care measures: antiplatelet therapy; blood pressure control; cholesterol screening and control; hemoglobin A1c screening and control; smoking status recorded; smoking cessation intervention; and body mass index. Performance generally remained flat for most of the measures while using paper-based health records. For seven of the nine measures, the observed performance declined slightly after EHR adoption and then rebounded to pre-EHR levels or increased to higher rates after 6 months. The research team hypothesizes that the rebound may be a result of office staff and providers becoming more accustomed to the EHR systems.

Provider surveys have identified that while practices may have electronic tools, they may not realize that they need assistance to learn to use them. One specific tool that practices have struggled to use is referral tracking. The project team has published a manuscript in the Journal of the American Medical Informatics Association on the reliability of EHR-derived quality data: "Validity Of EHR Derived Quality Measurement For Performance Monitoring."

Target Population: Adults, Inner City*, Medicaid, Medically Underserved, Safety Net

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

* This target population is one of AHRQ's priority populations.

Bringing High Performing Systems to Small Practices - 2010

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-270: Utilizing Health Information Technology to Improve Health Care Quality Grant (R18)
  • Grant Number: 
    R18 HS 018275
  • Project Period: 
    December 2009 – November 2012
  • AHRQ Funding Amount: 
    $1,199,853
  • PDF Version: 
    (PDF, 419.88 KB)


Target Population: Adults, Inner City*, Medicaid, Medically Underserved, Safety Net

Summary: To date, there is limited evidence on the ability of small community health care providers to improve quality of care through the use of electronic health records (EHRs), and limited data on the impact of financial incentives for quality improvement on small providers. Investments in health information technology (IT) are being made to improve quality of care and, while there is evidence of improved quality in integrated delivery systems, such as the Kaiser Permanente system, there is less evidence of the effectiveness of health IT on patient outcomes in nonintegrated health systems.

This study will provide information on the effects that supportive EHR implementation, clinical decision support (CDS) systems, and pay-for-quality improvements have on small community providers’ cardiovascular health outcomes. The New York City Primary Care Information Project (PCIP) is comparing the implementation of EHRs at 60 small ambulatory primary care practices that are not part of an integrated delivery system throughout New York City to 60 similar practices in the area that do not have an EHR. The project targets EHR implementation throughout New York City, with a focus on some of the poorest neighborhoods. The majority of practices are using Certification Commission Heath Information Technology-certified eClinical Works.

The study will evaluate the impact of an EHR implemented with the support of technical assistance, and added tools, including integrated registry systems and CDS, on improvements in quality of care as compared to practices that do not have an EHR or the aforementioned support programs. The primary goal is to determine whether practices that have supportive EHR implementation provide higher-quality care and experience a more rapid rate of improvement of their quality measures than practices that do not have an EHR. A secondary goal is to determine the characteristics, if any, that indicate supported EHR practices are atypical, or have any consistently different characteristics, as compared to other small independent practices. At a more nuanced level, the research will assess the attributable impact of various interventions on changes in four cardiovascular health outcomes at small practices that provide adult primary care. This will provide specific information on the value of various types of support on the rate of improvement on cardiovascular quality measures.

Specific Aims:
  • Determine whether practices that participated in the PCIP program experienced a more rapid rate of improvement on their quality measures than practices that did not participate. (Ongoing)
  • Determine if PCIP-participating practices are atypical in comparison to other small independent practices in New York City. (Ongoing)
  • Assess the attributable impact of each intervention: adoption of EHR, CDS, and pilot pay-for-quality program. (Upcoming)

2010 Activities: The project finalized the group of PCIP providers that is the comparison group for the active intervention practices that are receiving supportive EHR implementation. EHR adoption among small clinics in New York has moved rapidly since the writing of the grant, and there are fewer practices that have not begun EHR implementation. As a result, the project is shifting the definition of the “control” practices from non-EHR adopters to a subset of practices that are late adopters of the EHR. For the later adopters, 58 practices were recruited, representing a total of 134 providers. Similar to the early adopters, the majority of practices that were recruited as the late adoption group are solo or two-person practices. The project has decided to define early adopters as those that adopted an EHR prior to January 2009. Late adopters are those that adopted between January 2009 and March 2010.

The team developed and successfully implemented a baseline provider survey tool on their experiences with quality measurement, reporting, and incentives. A separate survey was distributed by PCIP as part of the overall regional extension center activities to assess the practice’s orientation and experience in completing tasks such as documentation and ordering. Practice characteristics, such as number of providers, ancillary staff, and patient demographics, are collected either through the practice’s application or through the chart review process.

To assess the impact of the interventions on quality, a number of quality measures will be collected through chart review. The team developed a form, database, and instruction set to collect the clinical data elements from paper charts that can be used to calculate the same quality measures as those calculated through the EHR. Chart review will continue through 2011. The study design was updated to include two time points prior to EHR adoption for the early EHR adopters for contemporaneous comparison with the later EHR adopters. This is needed to assess trends in quality measurement prior to EHR adoption. The two trends (pre-EHR vs. post-EHR) will enable the research team to determine which factors may be associated with changes in the trends in practice performance on quality measures. The team is also seeking additional data sources external to the PCIP program (e.g., health plan claims), to compare performance on quality metrics in these practices.

Grantee's Most Recent Self-Reported Quarterly Status (as of December 2010): Progress is completely on track. The team is on time on all tasks and the budget spending is on track.

Preliminary Impact and Findings: Data describing the characteristics of the early and late adopters is under internal review and will be submitted to the American Journal of Public Health. Preliminary analyses in 2010 for a few selected items focused on providers who received incentive payments and their perceptions of the payments. These analyses were used to gauge the motivation of providers in earning incentives tied to quality performance.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

*AHRQ Priority Population.

Bringing High Performing Systems to Small Practices - Final Report

Citation:
Parsons A. Bringing High Performing Systems to Small Practices - Final Report. (Prepared by New York City Health and Mental Hygiene under Grant No. R18 HS018275). Rockville, MD: Agency for Healthcare Research and Quality, 2013. (PDF, 253.56 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
Population: 
Medical Condition: 
This project does not have any related resource.

NYC REACH Regional Extension Center Provider Survey

This is a questionnaire designed to be completed by clinical staff in an ambulatory setting. The tool includes questions to assess perception of electronic health records.

Year of Survey: 
Created prior to 2012
Survey Link: 
NYC REACH Regional Extension Center Provider Survey (PDF, 296.58 KB) (Persons using assistive technology may not be able to fully access information in this report. For assistance, please contact Corey Mackison)
Document Type: 
Research Method: 
Population: 
Care Setting: 
Copyright Status: 
Permission has been obtained from the survey developers for unrestricted use of this survey; it may be modified or used as is without additional permission from the authors.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.