FluAlert: Influenza Vaccine Alerts for Providers in the Electronic Health Record (New York)

Project Final Report (PDF, 652.26 KB) Disclaimer

Project Details - Ended

Project Categories

Summary:

Influenza infection can result in substantial costs, morbidity, and mortality, all of which would be significantly reduced by increasing vaccination rates. Guidelines recommend that all children age 6 months and older receive the influenza vaccine. Despite this, vaccine delivery rates are low, and far below rates for other childhood vaccinations. Missed opportunities to vaccinate contribute to low coverage. Influenza vaccines are given yearly, but only during certain months, so providers must remember to offer vaccination at all visits during the influenza season—not just during well-child visits, as is done with other vaccines.

Clinical decision support in electronic health records (EHRs) can remind providers about adherence to practice guidelines. This project implemented and evaluated an EHR influenza vaccine alert, FluAlert, to improve pediatric providers’ decisionmaking around influenza vaccines.

The specific aims were to:

  • Integrate tailored provider influenza vaccine alerts into the EHRs of urban pediatric community health centers. 
  • Evaluate the impact of tailored provider influenza alerts on pediatric influenza vaccine delivery rates. 
  • Evaluate the impact of tailored provider influenza alerts on pediatric influenza coverage rates. 

Input into the design of the alert was obtained from focus groups and semi-structured interviews conducted with providers and parents. For those children not up-to-date with the influenza vaccine, providers received an alert that allowed them to order a vaccine or document the reason it was not ordered. FluAlert was evaluated with a cluster crossover study. Four urban academically affiliated community clinics that serve a primarily low-income Latino population participated. Two clinics were randomly assigned to begin the study with the FluAlert activated, and two began with the alert off. At 4 weeks the activation status was reversed between the clinics. Three main types of analyses were done: 1) process measures related to provider’s actions in response to the alert; 2) impact of alert on receipt of vaccination for children who were not up-to-date for the influenza vaccine; and 3) impact of alert on documentation of why a vaccine was not given.

FluAlert had a significant impact on influenza vaccination for children who were not up-to-date on their influenza vaccine. In addition, documentation in the EHR of why the vaccine was not ordered was significantly higher. The project team found that a tailored influenza vaccine alert in the EHR decreased missed opportunities for vaccination and improved documentation.

FluAlert: Influenza Vaccine Alerts for Providers in the Electronic Health Record - 2012

Summary Highlights

  • Principal Investigator: 
  • Organization: 
  • Funding Mechanism: 
    PAR: HS08-270: Utilizing Health Information Technology to Improve Health Care Quality Grant (R18)
  • Grant Number: 
    R18 HS 018158
  • Project Period: 
    August 2009 – July 2013
  • AHRQ Funding Amount: 
    $1,198,851
  • PDF Version: 
    (PDF, 343.96 KB)

Summary: The Advisory Committee on Immunization Practices recommends that all children age 6 months and older receive the influenza vaccine. Despite this recommendation, vaccine delivery rates are low, even when the vaccine is available. This project aimed to tailor, implement, and evaluate influenza vaccine alerts in an electronic health record (EHR) for pediatric providers serving minority low-income populations at four community health centers.

Each of the four study sites is affiliated with the New York-Presbyterian Hospital Ambulatory Care Network (ACN) and Columbia University and located in a federally designated Health Professional Shortage Area. All providers in the study are part of the same general pediatric group practice and receive uniform influenza vaccine-related provider education. In 2008, the practices had a volume of nearly 64,000 visits by approximately 22,000 children, the majority of whom were covered by Medicaid and were Latino. The Vaccine for Children Program provides most of the vaccines given at the practices.

In the first year of this project, Dr. Stockwell and her research team conducted focus groups and individual interviews with health care providers, nurses, and parents to elicit information for customizing the content, format, and features of the electronic alerts (FluAlert). In the second year, the alerts were iteratively refined and piloted among beta users based on end-user feedback. In the third year, the alerts were pilot tested within the four study sites using a cluster cross-over design. Throughout the study, the research team tracked process indicators such as user rates and obtained feedback from clinical sites. At the end of the project, user satisfaction will be assessed using survey data. Costs will be measured by comparing alert costs with published vaccine effectiveness rates and determining the costs for influenza-associated hospitalizations, outpatient visits, and impact on parent productivity.

Specific Aims:

  • Integrate tailored provider influenza vaccine alerts into the EHRs of urban pediatric community health centers. (Achieved)
  • Evaluate the impact of tailored provider influenza alerts on pediatric influenza vaccine delivery rates. (Ongoing)
  • Evaluate the impact of tailored provider influenza alerts on pediatric influenza coverage rates. (Ongoing)

2012 Activities: The FluAlert cluster cross-over trial began in October 2011 and continued until March 2012. Two clinics were randomly assigned to begin the study with the FluAlert activated and two clinics with it off. After 4 weeks, the activation status was reversed. The activation process was to be switched twice so that each clinic had two 4-week periods when FluAlert was active and two 4-week periods when it was off; each had an on and off period in the fall and winter. In this manner, the cluster cross-over design used each clinic as its own control group. Vaccine delivery performance is being compared during periods when the FluAlert was active to periods when it was off. After the trial was completed, Dr. Stockwell worked with a statistician to analyze the study data.

As last self-reported in the AHRQ Research Reporting System, project progress and activities are on track and the project budget funds are slightly underspent to conserve funds to complete the trial and subsequent data analysis.

Preliminary Impact and Findings: Analysis of the transcripts from focus groups with physicians identified several barriers to influenza vaccine delivery, including clinic resource issues, problems with multiple sources of immunization information, and lack of time to complete the vaccination process. The physicians also identified ways to improve the computerized reminder, such as timing of presentation, ability to access multiple sources of immunization records, and facilitation of vaccine ordering and documentation. These results were published in the March-April 2011 volume of Preventive Medicine in a manuscript titled FluAlert: a qualitative evaluation of providers’ desired characteristics and concerns regarding computerized inf luenza vaccination alert.

Focus groups were also held with parents to learn more about their experiences with the flu vaccine and their thoughts about how to improve communication with providers. Parents indicated the importance of hearing about both benefits and potential risks of the vaccine, especially when vaccine safety is a concern. Thematic analysis indicated that parents want to learn about their child’s risk for influenza and the side effects, safety, effectiveness, and timing of the vaccine.

In the 10-week pilot period, FluAlert generated 1,949 alerts. Fifty-four percent of the time (n=1,048), the alert indicated that the child was in need of an influenza vaccine. For those 1,048 instances, the vaccine was ordered 29 percent of the time; declined 34 percent of the time; and deferred 37 percent of the time. In cases when the vaccine was declined, the reason for declining the vaccine was documented 93 percent of the time. Children had greater odds of being vaccinated for flu when FluAlert was turned on than when it was turned off.

Analysis of the cluster cross-over trial is underway. The process evaluation showed that clinicians acted 82 percent of the time when a vaccine was required, two-thirds of those times they vaccinated. When vaccination did not occur, clinicians recorded a reason 98 percent of the time. Data analysis will continue in 2013.


Target Population: Low SES/Low Income*, Medicaid, Pediatric*, Racial or Ethnic Minorities*: Latinos, Teenagers

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

* This target population is one of AHRQ’s priority populations.

FluAlert: Influenza Vaccine Alerts for Providers in the Electronic Health Record - 2011

Summary Highlights

  • Principal Investigator: 
  • Organization: 
  • Funding Mechanism: 
    PAR: HS08-270: Utilizing Health Information Technology to Improve Health Care Quality Grant (R18)
  • Grant Number: 
    R18 HS 018158
  • Project Period: 
    August 2009 - July 2012
  • AHRQ Funding Amount: 
    $1,198,851
  • PDF Version: 
    (PDF, 204.51 KB)

Summary: The Advisory Committee on Immunization Practices recommends that all children age 6 months and older receive the influenza vaccine. Despite this recommendation, vaccine delivery rates are low, even when the vaccine is available. This project aimed to tailor, implement, and evaluate influenza vaccine alerts in an electronic health record (EHR) for pediatric providers serving minority low-income populations at four community health centers.

Each of the four study sites is affiliated with the New York-Presbyterian Hospital Ambulatory Care Network (ACN) and Columbia University and is located in a federally-designated Health Professional Shortage Area. All providers in the study are part of the same general pediatric group practice and receive uniform influenza vaccine-related provider education. In 2008, the practices had a volume of nearly 64,000 visits by approximately 22,000 children, the majority of whom were covered by Medicaid and were Latino. The Vaccine for Children Program provides most of the vaccines given at the practices.

In the first year of this project, Dr. Stockwell and her research team conducted focus groups, individual interviews, and surveys of health care providers, nurses, and parents to elicit information for customizing the content, format, and features of the electronic alerts (FluAlert). In the second year, the alerts were iteratively refined and piloted among beta users based on end-user feedback. In the third year, the alerts were pilot tested within the four study sites using a cluster cross-over design. Throughout the study, the research team tracked process indicators, such as user rates, and obtained feedback from clinical sites. At the end of the project, user satisfaction will be assessed using survey data. Costs will be measured by comparing alert costs with published vaccine effectiveness rates and determining the costs for influenza-associated hospitalizations, outpatient visits, and impact on parent productivity.

Specific Aims:

  • Integrate tailored provider influenza vaccine alerts into the EHRs of urban pediatric community health centers. (Achieved)
  • Evaluate the impact of tailored provider influenza alerts on pediatric influenza vaccine delivery rates. (Ongoing)
  • Evaluate the impact of tailored provider influenza alerts on pediatric influenza coverage rates. (Ongoing)

2011 Activities: The FluAlert pilot study that began August 2010 was finished in the beginning of 2011. Data from the pilot study was collected and analyzed. Following pilot testing, the research team gathered feedback from physicians and nurses to assess the functionality of FluAlert. Based on this feedback, the following system modifications were implemented: 1) every person who accessed the visit note received an alert, rather than only the individual who initiated the note; 2) print buttons were added to allow providers to print educational information about vaccination; 3) guidelines about egg allergies and thimerosal were updated; and 4) cosmetic changes were made to the graphic user interface.

The FluAlert cluster cross-over trial began in October 2011 and will continue until March 2012. Two clinics were randomly assigned to begin the study with the FluAlert activated and two clinics with it off. After 4 weeks, the activation status was reversed. The activation process will be switched twice so that each clinic will have two 4-week periods when FluAlert is active and two 4-week periods when it is off, not including holiday weeks. In this manner, the cluster cross-over design uses each clinic as its own control group. Vaccine delivery performance will be compared during periods when the FluAlert is active to periods when it is off.

As last self-reported in the AHRQ Research Reporting System, project progress and activities are mostly on track and the project budget funds are slightly underspent to conserve funds to complete the trial and subsequent data analysis.

Preliminary Impact and Findings: Analysis of the transcripts from focus groups with physicians identified several barriers to influenza vaccine delivery, including clinic resource issues, problems with multiple sources of immunization information, and lack of time to complete the vaccination process. The physicians also identified ways to improve the computerized reminder, such as timing of presentation, ability to access multiple sources of immunization records, and facilitation of vaccine ordering and documentation. These results were published in the March-April 2011 volume of Preventive Medicine.

Focus groups were also held with parents to learn more about their experiences with the flu vaccine and their thoughts about how to improve communication with providers. Parents indicated the importance of hearing about both benefits and potential risks of the vaccine, especially when vaccine safety is a concern. Thematic analysis indicates that parents want to learn about their child's risk for influenza and the side effects, safety, effectiveness, and timing of the vaccine.

In the 10-week pilot period, FluAlert generated 1,949 alerts. Fifty-four percent of the time (n=1,048), the alert indicated that the child was in need of an influenza vaccine. For those 1,048 instances, the vaccine was ordered 29 percent of-the time; declined 34 percent of the time; and deferred 37 percent of the time. In cases when the vaccine was declined, the reason for declining the vaccine was documented 93 percent of the time. Children had a 1.29-greater odds of being vaccinated for flu when FluAlert was turned on than when it was turned off (95 percent confidence interval: 1.13-1.48).

Target Population: Low SES/Low Income*, Medicaid, Pediatric*, Racial or Ethnic Minorities*: Latinos, Teenagers

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

* This target population is one of AHRQ's priority populations.

FluAlert: Influenza Vaccine Alerts for Providers in the Electronic Health Record - 2010

Summary Highlights

  • Principal Investigator: 
  • Organization: 
  • Funding Mechanism: 
    PAR: HS08-270: Utilizing Health Information Technology to Improve Health Care Quality Grant (R18)
  • Grant Number: 
    R18 HS 018158
  • Project Period: 
    August 2009 – July 2012
  • AHRQ Funding Amount: 
    $1,198,851
  • PDF Version: 
    (PDF, 330.79 KB)


Target Population: Low SES/Low Income*, Medicaid, Pediatric*, Racial or Ethnic Minorities*: Latinos,

Summary: The Advisory Committee on Immunization Practices recommends that all children age 6 months and older receive the influenza vaccine. Despite this recommendation, vaccine delivery rates at pediatric clinics are low, even when the vaccine is available. This project aims to tailor, implement, and evaluate influenza vaccine alerts in an electronic health record (EHR) for pediatric providers serving minority, low-income populations at four community health centers.

Each of the four study sites is affiliated with the New York-Presbyterian Hospital Ambulatory Care Network (ACN) and Columbia University and is located in a federally-designated Health Professional Shortage Area. All providers in the study are part of the same General Pediatric Group Practice and receive uniform influenza vaccine-related provider education. In 2008, the practices had a volume of nearly 64,000 visits by approximately 22,000 children, 87 percent of whom were covered by Medicaid and the majority of whom were Latino. The Vaccine for Children Program provides the majority of vaccines given at the practices. All four study sites use the Certification Commission for Health Information Technology-certified product Eclipsys Ambulatory Care Manager EHR and the New York Presbyterian Hospital Immunization Registry, EzVAC.

In Year 1, focus groups; individual interviews; and surveys of health care providers, nurses, and parents were conducted to elicit information for customizing the content, format, and features of the electronic alerts (FluAlert). The alerts were iteratively refined and piloted among beta users based on end-user feedback. In Year 2, the alerts began pilot testing within the four pediatric ACN community health centers of New York-Presbyterian Hospital using a cluster cross-over design. In Year 3, the alerts will be further assessed. Throughout the study period, process indicators will be tracked to follow the implementation of the system, and feedback with clinical sites will be regularly exchanged. At the end of the project, user satisfaction will be assessed through surveys. Cost will be measured by comparing alert costs with published vaccine effectiveness and cost data for influenza-associated hospitalizations, outpatient visits, and impact on parent productivity.

Specific Aims:
  • Integrate tailored provider influenza vaccine alerts into the EHRs of urban pediatric community health centers. (Achieved)
  • Evaluate the impact of tailored provider influenza alerts on pediatric influenza vaccine delivery rates. (Ongoing)
  • Evaluate the impact of tailored provider influenza alerts on pediatric influenza coverage rates. (Ongoing)

2010 Activities: The development of the FluAlert application and its integration into the EHR was completed this year. The FluAlert reminder functions with Eclipsys to retrieve immunization information from the EzVAC immunization registry, which is synchronized with New York City’s Department of Health Citywide Immunization Registry. This ensures that the alert is based on the most recent patient-specific influenza vaccine information. Additionally, FluAlert’s graphical user interface (GUI) was designed, evaluated, and revised to reflect feedback from the provider supervisory panel. The GUI alerts the provider to the patient’s influenza immunization status and allows the provider to order an influenza vaccine or document why the vaccine was not given. The alert also allows providers to access important clinical information, such as allergies and immunization history.

Following development of the FluAlert application, the research team met with physicians and nurses to present and improve FluAlert. In response to the information collected from these meetings, the research team added some non-core functions, including print buttons and visual effects. The team also improved upon non-functional aspects of FluAlert, such as performance time, security, and usability.

An end-to-end data transfer mechanism between FluAlert and Eclipsys was also developed. This data transfer mechanism allows FluAlert to pass information back to Eclipsys, where it is automatically included in the provider notes, and facilitates documentation of vaccinations.

Grantee's Most Recent Self-Reported Quarterly Status (as of December 2010): Overall the project is mostly on track and is meeting most of its aims and milestones. The project is on track but slightly underspent due to late distribution of funds.

Preliminary Impact and Findings: Analysis of the transcripts from focus groups with physicians about their experiences giving the flu vaccine identified several barriers to influenza vaccine delivery, including clinic resource issues, problems with multiple sources of immunization information, and lack of time to complete the vaccination process. They also identified ways to improve the computerized reminder, such as timing of presentation, ability to access multiple sources of immunization records, and facilitation of vaccine ordering and documentation.

Focus groups were also held with parents to learn more about their experiences with the flu vaccine and their thoughts about how to improve communication with providers. Parents indicated the importance of hearing about both benefits and potential risks of the vaccine, especially when vaccine safety is a concern. Preliminary thematic analysis indicates that parents want to learn about their child’s risk for influenza and the side effects, safety, effectiveness, and timing of the vaccine.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

*AHRQ Priority Population.

FluAlert: Influenza Vaccine Alerts for Providers in the Electronic Health Record - Final Report

Citation:
Stockwell M. FluAlert: Influenza Vaccine Alerts for Providers in the Electronic Health Record - Final Report. (Prepared by Columbia University under Grant No. R18 HS018158). Rockville, MD: Agency for Healthcare Research and Quality, 2013. (PDF, 652.26 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.