Implementation Outcomes of a Health Information Technology Program For Vulnerable Diabetes Patients (California)

Project Final Report (PDF, 332.07 KB) Disclaimer

Summary:

This study evaluated the Self-Management Automated Real-Time Telephone Support (SMART-Steps) program, which was developed through a previous Agency for Healthcare Research and Quality (AHRQ) grant (R18 HS 017261). SMART-Steps used an automated telephone self-management (ATSM) support system to provide monitoring and education for diabetic patients enrolled in the San Francisco Health Plan (SFHP). The focus of this more recent project was on examining the fidelity of the implementation of SMART-Steps to better-inform tailoring of health information technology interventions, particularly those focused on diverse populations and multiple languages.

The fidelity analysis focused on two primary areas:  health systems integration and moderating factors. The former included population-based data linkage to determine patient eligibility; electronic exchange of health information to deliver the intervention to patients; and electronic integration of health information to identify patients requiring a health coach callback or a callback for a medication or lab trigger. The moderating factors included representativeness of participants (reach); quality of intervention delivery in health coach callbacks (responsiveness); and consistency of delivery of the intervention over time.         

The specific aims of this project were to:

  • Estimate the proportion of patients identified as SMART-Steps-eligible who were ineligible, and describe reasons for ineligibility.
  • Determine if SMART-Steps patients received ATSM calls with intended frequency (weekly), content (questions/language), and duration (27 weeks).
  • Estimate the frequency with which electronic exchange for out-of-range triggers (from ATSM and SFHP clinical registry/pharmacy claims) resulted in a documented callback, in a sample of patients stratified by language.
  • Compare SMART-Steps-enrolled to -eligible patients for clinic, age, language, sex, hemoglobin A1c, insulin use, blood pressure, cholesterol, and prior medication non-adherence.
  • Describe the quality of intervention delivery from care managers’ callbacks, including frequency of supplemental self-management support, call duration, adherence to protocols, and creation of patient action plans, for a diverse sample of patient triggers.
  • Over the course of SMART-Steps implementation, identify differences in average length of callbacks, proportion of callbacks made for triggers, and whether wait-list patients (vs. not) had differential ATSM engagement.
  • Summarize fidelity assessment findings, adaptations, and implications for real-world ATSM implementation and related health IT interventions into a guide, with SFHP partnership.        

The assessment was conducted between 2009 and 2011 among 252 patients with type 2 diabetes. Fidelity to health systems integration was high. Eligibility data linkages were successful, with 76 percent of potential participants determined to be eligible. Ninety-six percent of patients received correctly sequenced ATSM calls; 70 percent of those requiring a callback received one from a health coach. There was a high level of variation in callbacks by type of patient problem and study language, warranting consideration for implementing health IT interventions in diverse populations.

Implementation Outcomes of a Health Information Technology Program For Vulnerable Diabetes Patients - 2012

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-268: Small Research Grant to Improve Health Care Quality through Health Information Technology (IT) (R03)
  • Grant Number: 
    R03 HS 020684
  • Project Period: 
    July 2011 – December 2012
  • AHRQ Funding Amount: 
    $100,000
  • PDF Version: 
    (PDF, 342.68 KB)

Summary: This study evaluated the Self-Management Automated Real-Time Telephone Support (SMART- Steps) program, which was developed through a previous Agency for Healthcare Research and Quality grant (R18 HS 017261). SMART-Steps used an automated telephone self-management (ATSM) support system to provide monitoring and education for diabetic patients enrolled in the San Francisco Health Plan (SFHP). The ATSM used health information technology (IT) to help patients manage their care outside traditional ambulatory settings, blending automated pre-recorded telephone queries and education with targeted ‘live’ telephone counseling by care managers. Care managers called if patients responded ‘out of range’ to a query, such as not having checked their blood sugar in the prior 7 days. Counseling focused on self-efficacy and self-management skills. The research team evaluated SMART-Steps’ effect on patient-centered outcomes, safety events, and measures from the Healthcare Effectiveness Data and Information Set amongst English-, Spanish-, and Cantonese-speaking diabetes patients. SMART-Steps provided a unique opportunity to examine the real-world implementation process for an evidence-based health IT intervention.

The objective of this study was to describe implementation fidelity—the degree to which the intervention is delivered as intended—for core ATSM intervention components. The core components were: 1) population- based data linkage to determine eligibility; 2) electronic exchange of health information to deliver ATSM queries to patients; 3) electronic integration of health information to identify patients requiring a call-back for an ATSM trigger; and 4) electronic integration of data to identify patients requiring a call-back for a medication or laboratory trigger. Additionally, the study described the potential impacts of moderating factors or barriers to implementation fidelity as well as adaptations of ATSM from planned to actual implementation. Moderating factors included representation of participants versus eligible patients as measured by demographics and baseline clinical measurements, quality of intervention delivery in call- backs, and consistency of delivery over time. The team reviewed findings and identified adaptations made during implementation to inform future scale-up efforts.

Specific Aims:

  • Estimate the proportion of patients identified as SMART-Steps-eligible who were ineligible, and describe reasons for ineligibility. (Achieved)
  • Determine if SMART-Steps patients received ATSM calls with intended frequency (weekly), content (questions/language), and duration (27 weeks). (Achieved)
  • Estimate the frequency with which electronic exchange for out-of-range triggers (from ATSM and SFHP clinical registry/pharmacy claims) resulted in a documented call-back, in a sample of patients stratified by language. (Achieved)
  • Compare SMART-Steps-enrolled to -eligible patients for clinic, age, language, sex, hemoglobin A1c, insulin use, blood pressure, cholesterol, and prior medication non-adherence. (Achieved)
  • Describe the quality of intervention delivery from care managers call-backs, including frequency of supplemental self-management support, call duration, adherence to protocols, and creation of patient action plans, for a diverse sample of patient triggers. (Achieved)
  • Identify differences in average length of call-backs, proportion of call-backs made for triggers, and whether wait-list patients (vs. not) had differential ATSM engagement. (Achieved)
  • Summarize fidelity assessment findings, adaptations and implications for real-world ATSM implementation and related health IT interventions into a guide, with SFHP partnership. (Achieved)

2012 Activities: The research team focused on the qualitative analysis of the implementation fidelity of the SMART-Steps program protocol. Due to challenges securing the time of a statistician, Dr. Handley used a 6-month no-cost extension. As last reported in the ARHQ Research Reporting System, progress was completely on track and spending was on target. This project was completed in December 2012.

Impact and Findings: To participate in the study, patients needed to meet the following criteria: diabetic; enrolled in the health plan; and English-, Cantonese-, or Spanish-speaking. Patients were screened based on these criteria using health plan records and the electronic medical record. The SMART-Steps program was offered as a health plan member benefit, but due to limited resources it was not possible to enroll all members.

Implementation fidelity was high. Of the 910 patients initially identified as eligible, 220 did not meet eligibility requirements indicating that eligibility linkages were successful. There was a high level of ATSM call delivery and integration in SFHP data systems, with 362 participants receiving calls from the ATSM system. Of the 298 patients completing one or more ATSM calls, 221 had at least one ATSM trigger that resulted in a coaching call back. This represented 1,980 coaching call backs with records in the project database, including 70 percent with a person contact described, and 30 percent with a note indicating that either no contact was made, a message was left, or an action was made on behalf of patient. There was a high level of variation in call backs by type of patient problem and by study language, which warrants consideration for implementing health IT interventions in diverse populations.

Among participants who were successfully contacted, coaching was extensive and the average call lasted approximately 10 minutes. Additionally, counseling was highly specific to the trigger event. An interesting aspect of the study was the impact of the patient-centered care on nonadherence coaching. Some of the nonadherence-triggered calls did not focus on nonadherence because patients were uncomfortable with the topic and turned the discussion in another direction. Because of the commitment to a patient-centered approach, coaches were instructed not to push conversations into areas where patients were not comfortable.

The SFHP staff who participated in the interviews reported an overall positive experience with the SMART- Steps program. The following protocol-fidelity barriers were identified: 1) due to high staff turnover during database development, many people contributed to the development process, which resulted in a slightly cumbersome user interface; 2) call managers reported that it was difficult to triage calls; and 3) due to the structure of the health plan, care managers were required to contact a designated “point person” in clinic if the patient had a serious issue; however, the call managers would have preferred to communicate directly with the provider.

Target Population: Adults, Chronic Care*, Diabetes, Elderly*, Low Literacy, Low-SES/Low Income*, Medicaid, Medically Underserved, Medicare, Racial or Ethnic Minorities*: Asian Cantonese and Hispanic Latino

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to support patient-centered care, the coordination of care across transitions in care settings, and the use of electronic exchange of health information to improve quality of care.

Business Goal: Knowledge Creation

*This target population is one of AHRQ’s priority populations.

Implementation Outcomes of a Health IT Program For Vulnerable Diabetes Patients - 2011

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    PAR: HS08-268: Small Research Grant to Improve Health Care Quality through Health Information Technology (IT) (R03)
  • Grant Number: 
    R03 HS 020684
  • Project Period: 
    July 2011 - June 2012
  • AHRQ Funding Amount: 
    $100,000
  • PDF Version: 
    (PDF, 200.4 KB)

Summary: This study is evaluating the Self-Management Automated Real Time Telephone Support (SMART-Steps) Program, which was developed through a previous Agency for Healthcare Research and Quality grant (R18 HS 017261). SMART-Steps used an automated telephone self-management (ATSM) support system to provide monitoring and education diabetic patients enrolled in the San Francisco Health Plan (SFHP). ATSM used health information technology (IT) to help patients self-manage outside traditional ambulatory settings, blending automated pre-recorded telephone queries and education with targeted 'live' telephone counseling by care managers. Care managers called if patients responded 'out of range' to a query, such as not having checked their blood sugar in the past 7 days. Counseling focused on self-efficacy and self-management skills. The research team is evaluating SMART-Steps Program's effect on patient-centered outcomes, safety events, and measures from the Healthcare Effectiveness Data and Information Set amongst English-, Spanish-, and Cantonese-speaking diabetes patients. SMARTSteps provides a unique opportunity to examine the real-world implementation process for an evidencebased health IT intervention.

The objective of this study is to describe implementation fidelity - the degree to which the intervention is delivered as intended - for core ATSM intervention components. The core components were: 1) population-based data linkage to determine eligibility; 2) electronic exchange of health information to deliver ATSM queries to patients; 3) electronic integration of health information to identify patients requiring a call-back for an ATSM trigger; and 4) electronic integration of data to identify patients requiring a callback for a medication or laboratory trigger. Additionally, the study will describe the potential impacts of moderating factors, or barriers to implementation fidelity as well as adaptations of ATSM from planned to actual implementation. Moderating factors will include representation of participants versus eligible patients as measured by demographics and baseline clinical measurements, quality of intervention delivery in call-backs, and consistency of delivery over time. The team will review findings and identify adaptations made during implementation to inform future scale-up efforts and create an ATSM implementation guide for dissemination. This information will help interpret results from health IT interventions and guide adaptation and scale-up activities by organizations undertaking similar programs.

Specific Aims:

  • Estimate the proportion of patients identified as SMART-Steps-eligible who were ineligible, and describe reasons for ineligibility. (Achieved)
  • Determine if SMART-Steps patients received ATSM calls with intended frequency (weekly), content (questions/language), and duration (27 weeks). (Achieved)
  • Estimate the frequency with which electronic exchange for out-of-range triggers (from ATSM and SFHP clinical registry/pharmacy claims) resulted in a documented call-back, in a sample of patients stratified by language. (Ongoing)
  • Compare SMART-Steps-enrolled to -eligible patients for clinic, age, language, sex, hemoglobin A1c, insulin use, blood pressure, cholesterol, and prior medication non-adherence. (Achieved)
  • Describe the quality of intervention delivery from care managers call-backs, including frequency of supplemental self-management support, call duration, adherence to protocols, and creation of patient action plans, for a diverse sample of patient triggers. (Ongoing)
  • Over the course of SMART-Steps implementation, identify differences in average length of callbacks, proportion of call-backs made for triggers, and whether wait-list patients (vs. not) had differential ATSM engagement. (Upcoming)
  • Summarize fidelity assessment findings, adaptations and implications for real world ATSM implementation and related health IT interventions into a guide, with SFHP partnership. (Ongoing)

2011 Activities: The research team's focus in 2011 was on the quantitative and qualitative analysis of the implementation of fidelity of the SMART-Steps Program protocol. They analyzed study enrollment data to determine which patients participated, declined, or were not contacted due to limited resources, including unable to contact or not eligible as determined by screening. Comparison of demographic and clinical characteristics by these groups allowed the study team to look for differences in participation rates. Additionally, an analysis of care manager telephone call records was conducted to determine the frequency with which the care manager was able to speak with the SMART-Steps participant. Finally, the study team conducted interviews with the SFHP staff to identify facilitators and barriers of following the research protocol.

As last self-reported in the AHRQ Research Reporting System, project progress and activities are on track and project budget spending is roughly on target.

Preliminary Impact and Findings: To participate in the study, patients needed to meet the following criteria: diabetic, enrolled in the health plan; and English-, Cantonese-, or Spanish-speaking. Patients were screened based on these criteria using the health plan records and the electronic medical record. The SMART-Steps Program was offered as a health plan member benefit, but due to limited resources it was not possible to enroll all members. The analysis of members who declined study participation or were not contacted because of limited resources indicated that there were no differences in demographic characteristics. The call manager telephone data showed that more than 95 percent of calls were correctly delivered. For calls that were not correctly delivered, one reason included calls not going out to newly enrolled participants.

The SFHP staff who participated in the interviews reported an overall positive experience with the SMART-Steps Program. The following barriers to protocol fidelity were identified: 1) due to high staff turnover during database development, many people contributed the development process, which resulted in a slightly cumbersome user interface; 2) call managers reported that it was difficult to triage calls; and 3) due to the structure of the health plan, care managers were required to contact a designated "point person" in clinic if the patient had a serious issue; however, the call managers would have preferred to communicate directly with the provider.

Target Population: Acute Respiratory Infections, Adults, Chronic Care*, Diabetes, Elderly*, Low Literacy, Low-SES/Low Income*, Medicaid, Medically Underserved, Medicare, Racial or Ethnic Minorities*: Cantonese and Spanish-speaking

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to support patient-centered care, the coordination of care across transitions in care settings, and the use of electronic exchange of health information to improve quality of care.

Business Goal: Knowledge Creation

* This target population is one of AHRQ's priority populations.

Implementation Outcomes of a Health Information Technology Program For Vulnerable Diabetes Patients - Final Report

Citation:
Handley M. Implementation Outcomes of a Health Information Technology Program For Vulnerable Diabetes Patients - Final Report. (Prepared by University of California - San Francisco under Grant No. R03 HS020684). Rockville, MD: Agency for Healthcare Research and Quality, 2012. (PDF, 332.07 KB)

The findings and conclusions in this document are those of the author(s), who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
Principal Investigator: 
Document Type: 
Medical Condition: 
This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.