An Evaluation of an Interactive Social Media Web site for Parents who are Concerned about Immunizing their Children (California)

Project Final Report (PDF, 1.45 MB) Disclaimer

Project Details - Ended

Project Categories

Summary:

Widespread use of immunizations in the United States has led to the eradication or control of numerous vaccine-preventable diseases including smallpox, polio, diphtheria, and measles. However, as the incidence of vaccine-preventable diseases has declined, public concern has shifted from disease transmission to vaccine safety. Many Web sites – both pro- and anti-vaccination – use social media to convey the benefits and risks of vaccinations. Numerous anti-vaccination Web sites disseminate erroneous information, discredit the medical community, and create fear about vaccinations. Often these sites are not moderated by experts, and the sources of health information are anonymous. New and expecting parents are typically active users of social media, which puts them at risk of finding these sites and believing the erroneous information.

This project built and evaluated an interactive social media Web site devoted to vaccines. The site has a forum in which parents can access truthful, unbiased information, and discuss vaccine-related issues with other parents and pediatricians. The participants in this study were active Kaiser Permanente Colorado members.

The specific aims of this project were to:

  • Design and develop an interactive social media Web site devoted to immunization. 
  • Conduct a qualitative formative evaluation of the social media Web site using focus groups. 
  • Qualitatively and quantitatively evaluate Web site usability through one-on-one testing sessions with end users. 
  • Pilot test the social media Web site with a representative cohort of end users over a 6-month followup period. 

In the first phase of the project, the Web site was designed and an initial prototype built. The objective was to create a resource that was both easy to use and interactive, with a blog, discussion forum, chat room, and an anonymous portal through which parents were able to ask questions. In the second phase of the project, focus groups and interviews were conducted with parents and providers to generate qualitative data to further develop and refine the Web site. Usability testing sessions with eight parents and two providers were conducted in Phase 3. In Phase 4, the Web site was modified based on the results of the focus groups, interviews, and usability testing to improve its functionality. In the final phase of the project, a survey with 443 parents and pregnant women was conducted to assess the vaccine decisionmaking process and preferences for vaccine information. The pilot test was completed with 86 participants.

The project team found that vaccine-hesitant parents were more likely to begin thinking about vaccines early, indicating that pregnancy is an important decisionmaking time for parents who are considering delaying or refusing vaccines. In addition, vaccine-hesitant parents indicated that they constantly re-evaluate their vaccine decisions. Participants used the prototype between zero and three times over a 6-month period. The project team concluded that an interactive social media Web site has the potential to help parents address vaccination concerns. Such an intervention, however, poses numerous challenges, including establishing trust, making it cost-effective, and recruiting new and expecting parents whose time is limited. The team is conducting a follow-on AHRQ-funded randomized intervention trial to further evaluate this tool and address these challenges.

An Evaluation of an Interactive Social Media Web site for Parents who are Concerned about Immunizing their Children - 2012

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-269: Exploratory and Developmental Grant to Improve Health Care Quality Through Health Information Technology (IT) (R21)
  • Grant Number: 
    R21 HS 019760
  • Project Period: 
    September 2010 – September 2013
  • AHRQ Funding Amount: 
    $165,301
  • PDF Version: 
    (PDF, 258.03 KB)

Summary: Immunizations are one of the most significant public health achievements of the 20th century, preventing more than 2 million deaths per year worldwide. However, as the incidence of vaccine-preventable diseases has declined, public concern has shifted from disease transmission to vaccine safety. An increasing number of parents in developed countries now believe the risks of vaccines outweigh the benefits. Research has shown that parents who decline or delay immunizations greatly increase their children’s risk of pertussis (whooping cough), varicella (chicken pox), and measles infections. Research also shows that the health information that vaccine-hesitant parents obtain from the Internet is often inaccurate and biased.

Effective intervention strategies are needed to reduce parental concerns about immunizations. The objective of this study is to develop and evaluate an interactive social media Web site for parents who have concerns about vaccines. The Web site contains high-quality educational vaccine information and features several interactive features, including a vaccine scheduler, blog, discussion forum, and chat room. It is also a resource for providers who are interested in obtaining information about the latest vaccine-related concerns and for parents who want to discuss vaccine-related topics with vaccine experts, providers, and other parents.

The interactive social media Web site was developed by Dr. Glanz and his multidisciplinary team. The study was conducted in four phases. In the first phase, the team developed the vaccine content for the Web site, which includes information on vaccine-preventable diseases, how vaccines work, how vaccines are made, vaccine safety, vaccine laws, and common parental concerns about vaccinations. In the second phase, focus groups were conducted among parents who refused, delayed, or accepted childhood vaccinations to receive feedback on the Web site’s design, layout, and content. In the third phase, several usability testing sessions were conducted with parents to assess the functionality of the Web site. In the final phase, the use of the Web site will be qualitatively and quantitatively assessed over time and will include a longitudinal assessment of the pilot cohort’s knowledge, attitudes, and beliefs (KAB) about immunizations. This pilot investigation will inform a larger integrated behavioral health intervention to reduce parental concerns about vaccinations and increase immunization rates over time.

Specific Aims:

  • Design and develop an interactive, social media Web site devoted to immunizations. (Achieved)
  • Conduct a qualitative, formative evaluation of the social media Web site using focus groups. (Achieved)
  • Qualitatively and quantitatively evaluate Web site usability through one-on-one testing sessions with end users. (Ongoing)
  • Pilot test the social media Web site with a representative cohort of end users over a 6-month followup period. (Ongoing)

2012 Activities: Dr. Glanz gave a number of presentations about the Web site. Audiences included the Immunization Task Force as well as Kaiser Foundation Research Institute leadership. The Web site was well-received by all audiences. Physicians in particular liked that the intervention reduced their work by allowing them to refer patients to the Web site for vaccine information.

The survey instrument to assess parental vaccine decisionmaking and KAB was administered to 443 parents who accepted, delayed, or refused vaccines for their children. All of the parents were enrolled in the Kaiser Permanente Colorado health plan.

The recruitment process for the pilot study began in July 2012 with a letter explaining the study to new and expecting parents. The study team found that multiple calls were needed to reach the target audience, given the busy lives of expecting parents. Once reached however, most were interested and agreed to participate.

To facilitate study recruitment, Dr. Glanz and his study team developed an online consenting mechanism that was approved by the institutional review board. The mechanism includes several steps: 1) participant identification; 2) identity confirmation; 3) application of study inclusion criteria; 4) consent; and 5) online survey completion. A challenge during the development process was how to collect and store identifying information (name and email address) required for the consent process, while protecting the personal information of a patient who has not yet consented to participate in the study. To solve this problem, the team used an encryption process that scrambles the personal information of potential participants and generates a unique identifier. Dr. Glanz emphasized the importance of thoroughly considering all aspects of online patient consent to streamline the process as much as possible to encourage study participation.

As last self-reported in the AHRQ Research Reporting System, project progress is on track in some respects but not others. Due to the additional work required to build the online consenting mechanism, Dr. Glanz is using a 1-year no-cost extension. Project spending is on target.

Preliminary Impact and Findings: The feedback from the focus groups was positive. All participants liked the Web site, regardless of whether they refused, delayed, or accepted vaccines for their children. Participants reported that the tone of the site was inclusive, informative, and unbiased. Several participants remarked that they liked that the funder of the Web site was not a private organization, nor an organization known for pushing a particular vaccine agenda. Participants appreciated the information explaining the diseases, and some even requested additional information. The interactive vaccine scheduler and a video that features two married pediatricians discussing vaccination for their child were also popular features. Participants felt that the information explaining aluminum levels in vaccines was not clear, and that there should be more photographs of fathers. Feedback from the focus groups was used to modify the Web site. The results were accepted for publication in Academic Pediatrics, with an anticipated publication date of October 2013.

Target Population: Other Conditions: Pertussis, Varicella, Measles, Pediatric*

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

*This target population is one of AHRQ’s priority populations.

An Evaluation of an Interactive Social Media Web site for Parents who are Concerned about Immunizing their Children - 2011

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-269: Exploratory and Developmental Grant to Improve Health Care Quality Through Health Information Technology (IT) (R21)
  • Grant Number: 
    R21 HS 019760
  • Project Period: 
    September 2010 - September 2012
  • AHRQ Funding Amount: 
    $165,301
  • PDF Version: 
    (PDF, 198.19 KB)

Summary: Immunizations are one of the most significant public health achievements of the 20th century, preventing more than 2 million deaths per year worldwide. However, as the incidence of vaccinepreventable diseases has declined, public concern has shifted from disease transmission to vaccine safety. An increasing number of parents in developed countries now believe the risks of vaccines outweigh their benefits. Research has shown that parents who decline or delay immunizations greatly increase their children's risk of pertussis (whooping cough), varicella (chicken pox), and measles infections. Research also shows that the health information that vaccine-hesitant parents obtain from the Internet is often inaccurate and biased.

Effective intervention strategies are needed to reduce parental concerns about immunizations. The objective of this study is to develop and evaluate an interactive social media Web site for parents who have concerns about vaccines. The Web site features an interactive vaccine scheduler as well as various social media applications including a blog, a discussion forum, and a chat room. It is also a resource for providers who are interested in obtaining information about the latest vaccine-related concerns and discussing vaccine-related topics with other providers.

The Web site will be piloted among patients and physicians of Kaiser Permanente Colorado (KPCO), a closed-panel, group-model health maintenance organization that provides integrated health care services to the Denver-Boulder Metropolitan Area. The Web site will be moderated by physicians and vaccine researchers at the KPCO Institute for Health Research. Use of the Web site will be qualitatively and quantitatively assessed over time and will include a longitudinal assessment of the pilot cohort's knowledge, attitudes, and beliefs about immunizations. This pilot investigation will inform future research to implement a larger, integrated behavioral health intervention to reduce parental concerns about vaccinations and increase immunization rates.

Specific Aims:

  • Design and develop an interactive, social media Web site devoted to immunizations. (Ongoing)
  • Conduct a qualitative, formative evaluation of the social media Web site using focus groups. (Achieved)
  • Qualitatively and quantitatively evaluate Web site usability through one-on-one testing sessions with end users. (Ongoing)
  • Pilot test the social media Web site with a representative cohort of end users over a 6-month followup period. (Upcoming)

2011 Activities: The interactive social media Web site was developed. Vandiver Group, Inc., built the Web site platform, and Dr. Glanz and his team wrote the Web site content. Significant consideration was given to deciding on the tone of the Web site. Dr. Glanz took a pro-information approach that includes validating concerns about vaccination and providing high-quality educational information. While the Web site is inherently pro-vaccine, Dr. Glanz did not want it to be so overt that undecided parents would be turned off.

Focus groups were conducted among parents who refused, delayed, or accepted vaccines for their children. Feedback from the focus groups was analyzed and the Web site is being modified to reflect the input from the focus group participants. One such modification was the addition of a table of all vaccines used by KPCO and the ingredients in those vaccines. The research team has received several requests from KPCO pediatricians to use the ingredient table in their practices, so the practices can share copies of the table with their patients. The research team also created a video that is prominently featured on the Web site. The video features two KPCO pediatricians who are married and have children and discuss childhood vaccination.

As last self-reported in the AHRQ Research Reporting System, project progress and activities are mostly on track and project budget spending is roughly on target. They are slightly behind due to initial delays with Web site development, but do not expect this to affect their overall project deadlines.

Preliminary Impact and Findings: The feedback from the focus groups was very positive. All participants liked the Web site, regardless of whether they refused, delayed, or accepted vaccines for their children. Participants reported that the tone of the Web site was inclusive, informative, and unbiased. Several participants remarked that they were happy to see that the money did not come from a private organization, or an organization known for pushing a particular vaccine agenda. Participants appreciated the information explaining the diseases, and some even requested additional information. The interactive vaccine scheduler and the video were also popular features of the Web site. Criticisms included that participants felt that the interactive tool that explains aluminum levels present in vaccines was not clear, and they wanted to see more photographs of fathers on the Web site. Parents wanted additional information about vaccines as well as clearly cited references.

Target Population: Other Conditions: Pertussis, Varicella, Measles, Pediatric*

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

* This target population is one of AHRQ's priority populations.

An Evaluation of an Interactive Social Media Web site for Parents who are Concerned about Immunizing their Children - 2010

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-269: Exploratory and Developmental Grant to Improve Health Care Quality Through Health Information Technology (IT) (R21)
  • Grant Number: 
    R21 HS 019760
  • Project Period: 
    September 2010 – September 2012
  • AHRQ Funding Amount: 
    $165,301
  • PDF Version: 
    (PDF, 312.27 KB)


Target Population: Adults, Other Conditions: Pertussis, Varicella, Measles

Summary: Immunizations are one of the most significant public health achievements of the 20th century, preventing more than 2 million deaths per year worldwide. However, as the incidence of vaccine- preventable diseases has declined, public concern has shifted from disease transmission to vaccine safety. An increasing number of parents in developed countries now believe the risks of vaccines outweigh their benefits. Research has shown that parents who decline or delay immunizations greatly increase their children’s risk of pertussis (whooping cough), varicella (chicken pox), and measles infections. Research also shows that the health information that vaccine-hesitant parents obtain from the Internet is often inaccurate and biased.

Effective intervention strategies to reduce parental concerns about immunizations are needed. The objective of this study is to develop and evaluate an interactive, social media Web site for parents who are concerned about vaccines. The Web site will feature various social media applications, including a blog, a discussion forum, and a social networking service. It will also be a resource for providers who are interested in obtaining information about the latest vaccine-related concerns and discussing vaccine-related topics with parents.

The Web site will be piloted among patients and physicians of Kaiser Permanente Colorado (KPCO), a closed-panel, group-model health maintenance organization that provides integrated health care services to the Denver-Boulder metropolitan area. The Web site will be moderated by physicians and vaccine researchers at the KPCO Institute for Health Research. Use of the Web site will be qualitatively and quantitatively assessed over time and will include a longitudinal assessment of the pilot cohort’s knowledge, attitudes, and beliefs about immunizations. This pilot investigation will inform future research to implement a larger, integrated behavioral health intervention to reduce parental concerns about vaccinations and increase immunization rates.

Specific Aims:
  • Design and develop an interactive, social media Web site devoted to immunizations. (Ongoing)
  • Conduct a qualitative, formative evaluation of the social media Web site using focus groups. (Ongoing)
  • Qualitatively and quantitatively evaluate Web site usability through one-on-one testing sessions with end users. (Ongoing)
  • Pilot test the social media Web site with a representative cohort of end users over a 6-month followup period. (Upcoming)

2010 Activities: The project was initiated in September 2010. Institutional Review Board approval was obtained to conduct focus groups and usability testing of the Web site. The development of the interactive social media Web site is in progress. Vandiver Group, Inc. is building the Web site and KPCO is writing the content.

Grantee's Most Recent Self-Reported Quarterly Status (as of December 2010): The project is meeting all milestones on time and is somewhat under-budget to conserve funds for later cost intensive activities.

Preliminary Impact and Findings: This project has no findings to date.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to improve health care decisionmaking through the use of integrated data and knowledge management.

Business Goal: Knowledge Creation

An Evaluation of an Interactive Social Media Web site for Parents who are Concerned about Immunizing their Children - Final Report

Citation:
Glanz J. An Evaluation of an Interactive Social Media Web site for Parents who are Concerned about Immunizing their Children - Final Report. (Prepared by Kaiser Foundation Research Institute under Grant No. R21 HS019760). Rockville, MD: Agency for Healthcare Research and Quality, 2013. (PDF, 1.45 MB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
Technology: 
Population: 
Medical Condition: 
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.