Evaluation of Computer Generated After-Visit Summaries to Support Patient-Centered Care (New Mexico)

Project Details - Ended

Project Categories

Summary:

Following a medical encounter, it is common practice to give patients educational materials and printed instructions in the form of handouts and pamphlets. Many electronic medical records (EMRs) are capable of printing an after-visit summary (AVS) that includes similar content as well as a patient-specific summary of the visit. AVSs have the potential to improve shared decisionmaking, coordination of care, and adherence to recommended plans of care. Content of the summary varies but generally includes the patient’s medications, diagnoses, problems, allergies, and instructions. Little research has been done on how patients use the AVS and whether it is an effective way of conveying information to patients. This project utilized input from patients and clinicians in order to develop three test versions of an AVS, and employed a randomized controlled trial to evaluate the various versions against a standard AVS.

The main objectives of the project were to:

  • Ascertain patient attitudes, preferences, and needs regarding the delivery of information at a visit with a primary care physician.
  • Identify primary physicians’ attitudes about the utility, content, and value of the AVS.
  • Develop and test three different versions of an AVS.
  • Disseminate the programming instructions needed to deploy an AVS for health care organizations that use the Epic EMR system.

The project was conducted in four clinics with a diverse payer mix, English and Spanish-speakers, and African-American, Hispanic, and non-Hispanic white clients. The same EMR was used at all four sites, although it was configured differently at each. Interviews were conducted with 12 providers and 48 of their patients in order to inform the content of the three AVS test versions. One of the three versions met Meaningful Use requirements, while the remaining two contained less information.

A total of 68 patients participated in the evaluation phase, with equal numbers of Spanish and English speakers. Two phone interviews were conducted: the first between1-3 days after a visit and the second between 14-21 days after. The purpose of the interviews was to assess the patient’s general AVS content recall, medication recall, satisfaction with the AVS, and self-reported adherence to treatment.

Recall of content of the AVS was low, with the highest being the group who received the shortest AVS version. Although health literacy was lower in the Spanish-speaking population, the study team did not find any significant difference in recall of medications between the groups. Overall, patients were happy with the format of the AVS, and indicated that the medication list and instructions were the most important components. Not surprisingly, Spanish-speaking patients indicated their preference for an AVS in their own language. The mean adherence score did not differ between groups.

The project team concluded that the patients liked to receive an AVS, but the amount and range of the included information did not impact their recall of health-related visit information, or their own perception of the information’s usefulness. However, given the low levels of recall of content it is likely that much of the information on the AVS is not retained by patients. The team concluded that the AVS must be supplemented with efforts to directly remind patients about important information and to not rely solely on the summary.

Evaluation of Computer Generated After-Visit Summaries to Support Patient-Centered Care - 2012

Summary Highlights

  • Principal Investigator: 
  • Funding Mechanism: 
    Primary Care Practice-Based Research Network (PBRN)
  • Contract Number: 
    290-07-10007-2
  • Project Period: 
    September 2009 – March 2012
  • AHRQ Funding Amount: 
    $496,788
  • PDF Version: 
    (PDF, 216.06 KB)

Summary: The office visit remains a cornerstone of primary care and health information delivery. Yet studies have shown that by the time the patient leaves a facility, he or she may forget as much as 50 percent of the information relayed during the visit, which can negatively affect the patient’s care.

Electronic medical records (EMRs) can provide patients with information about their clinical visits through personalized handouts that summarize the topics and recommendations covered during the visit. These after-visit summaries (AVS) have the potential to improve a patient’s retention of information and followup instructions, leading to better adherence to treatment plans. An AVS can also facilitate the transfer of information between health settings. However, the content and formatting of an AVS that will optimize patients’ information retention and satisfaction is still unknown.

Through a contract with the University of New Mexico, Dr. Valory Pavlik and her team from the Baylor College of Medicine, Department of Family Practice, provided expertise and guidance throughout the term of the project.

This project employed qualitative methods to gather patient and physician input into the AVS development. The research team then developed three AVS versions with varying amounts of included information. The three experimental versions were evaluated in a randomized trial to examine the effects of differences in information content on patient recall and satisfaction. A fourth group of patients received the standard AVS used in each clinic. The study took place in four clinics in Texas that use a common EMR and serve an ethnically and socioeconomically diverse patient population. The research team included experts in study design and analysis, medical informatics, bilingual and bicultural patient information transfer, qualitative analysis, and clinical practice-based research.

Project Objectives: 

  • Ascertain patient attitudes, preferences, and needs regarding the delivery of information at a visit with a primary care physician. (Achieved) 
  • Identify primary physicians’ attitudes about the utility, content, and value of the AVS. (Achieved)  
  • Develop and test three different versions of an AVS. (Achieved) 
  • Prepare an implementation guide to assist practices in developing and implementing an AVS. (Achieved) 

2012 Activities: Final data collection and analysis were completed during this period. The project team developed an implementation guide for dissemination. The project ended in March 2012.

Impact and Findings: A total of 272 patients participated in the evaluation phase, with equal numbers of Spanish and English speakers. Two phone interviews were conducted: the first between 1-3 days after a visit, and the second between 14-21 days after. The purpose of the interviews was to assess the patient’s general AVS content recall, medication recall, satisfaction with the AVS, and self-reported adherence to treatment.

Recall of content of the AVS was low, with the highest proportion of information recalled in the group who received the shortest AVS version. However, recall of individual components that were common across AVS versions did not differ among the group. Although health literacy was lower in the Spanishspeaking population, the team did not find any significant difference in recall of medications between the groups. Overall, patients were happy with the format of the AVS, and indicated that the medication list and instructions were the most important components. Not surprisingly, Spanish-speaking patients indicated their preference for an AVS in Spanish. The mean adherence score did not differ between groups.

The project team concluded that patients liked receiving an AVS, but the amount and range of the included information did not impact their recall of health-related visit information or their perception of the information’s usefulness. However, given the low levels of recall of content, it is not likely that patients retain much of the information on the AVS. The team concluded that the AVS must be supplemented with efforts to directly remind patients about important information and not rely solely on the summary.

Target Population: Adults

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to support patient-centered care, the coordination of care across transitions in care settings, and the use of electronic exchange of health information to improve quality of care.

Business Goal: Knowledge Creation

Evaluation of Computer Generated After-Visit Summaries to Support Patient-Centered Care - 2011

Summary Highlights

  • Principal Investigator: 
  • Contract Number: 
    290-07-10007-2
  • Project Period: 
    September 2009 - March 2012
  • AHRQ Funding Amount: 
    $496,788
  • PDF Version: 
    (PDF, 175.64 KB)

Summary: The office visit remains a cornerstone of primary care and health information delivery. Yet studies have shown that by the time the patient leaves a facility, he or she may forget as much as 50 percent of the information relayed during the visit, which can negatively affect a patient's care.

Electronic medical records (EMRs) offer a new method of providing patients with information about their clinical visits through personalized, patient-specific handouts that summarize the topics and recommendations covered during the visit. These after-visit summaries (AVS) have the potential to improve a patient's retention of information about adherence to treatment plans and followup instructions. AVS can also facilitate the transfer of information between health settings. However, the content and formatting of AVS that will optimize patients' information retention and satisfaction is still unknown.

Through a contract with the University of New Mexico, Dr. Valory Pavlik and her team from the Baylor College of Medicine, Department of Family Practice provided expertise and guidance throughout the term of the project.

This project employed qualitative methods to gather patient and physician input into AVS development. The research team then developed three AVS versions with varying amounts of included information. The three experimental versions were evaluated in a randomized trial to examine the effects of differences in information content on patient recall and satisfaction. A fourth group of patients received the standard AVS used in each clinic. The study took place in four clinics in Texas that use the EpicCare EMR and serve an ethnically and socioeconomically diverse patient population. The research team included experts in study design and analysis, medical informatics, bilingual and bicultural patient information transfer, qualitative analysis, and clinical practice-based research.

The research team will draft and submit a manuscript about the research effort to a peer-reviewed journal and will propose processes for dissemination, including publicity through primary care organizations, academic primary care departments, practice associations, and various organizations active in health information technology (IT) development. Technical findings and after-visit summary-related products will be available for download on the Internet for use by other primary care providers. The team will prepare a final report that includes all study deliverables (e.g., copies of all research instruments), results, conclusions, suggestions for additional research, and actionable lessons learned.

Project Objectives:

  • Ascertain patient attitudes, preferences, and needs regarding the delivery of information at a visit with a primary care physician. (Achieved)
  • Identify primary physicians' attitudes about the utility, content, and value of the AVS. (Achieved)
  • Develop and test three different versions of an AVS. (Achieved)
  • Prepare an implementation guide to assist practices in developing and implementing an AVS. (Ongoing)

2011 Activities: During the year, the research team reviewed a summary of key themes from patient and physician interviews that pertain to AVS format and content, and developed three different prototype AVS forms. Issues related to facilities, extreme weather, personnel, and information technology have slowed patient recruitment, but recruitment and data collection will be completed by March 2012.

Preliminary Impact and Findings: The research team completed data collection at the Baylor Family Medicine (BFM) clinic, Houston Service Workers Clinic (HSWC), and Harris County Hospital District (HCHD) Martin Luther King clinic. Fifty-six patients were recruited at BFM, 48 of whom received the first call-back and eight of whom missed it. Thirty-nine received the second call-back while nine were lost to followup. Eighteen patients declined to participate. At HSWC, 77 patients were recruited. Sixty-eight received the first call back and nine missed it. Fifty-three received the second call-back and 15 were lost to followup. Twelve patients refused to participate. At HCHD, 51 patients were recruited. Forty-eight received the first call-back, three missed it. Thirty-nine received the second call-back and nine were lost to followup. Ten patients refused to participate.

Study staff was in the process of collecting data at the Harris County Hospital District, Strawberry clinic. Their goal is 40 English-speaking and 68 Spanish-speaking patients. By the end of 2011, their results included:

  • English-speaking patients: Forty-four patients were recruited. Thirty-eight received the first call-back and zero missed it. No patients received the second call-back and none were lost to followup. Eight patients declined to participate.
  • Spanish-speaking patients: Fifty-four patients were recruited. Forty-nine received the first call-back; zero missed it. No patients received the second call-back and none were lost to followup. Six patients declined to participate.

Target Population: Adults

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to support patient- centered care, the coordination of care across transitions in care settings, and the use of electronic exchange of health information to improve quality of care.

Business Goal: Knowledge Creation

Evaluation of Computer Generated After Visit Summaries to Support Patient-Centered Care - 2010

Summary Highlights

  • Principal Investigator: 
  • Contract Number: 
    290-07-10007-2
  • Project Period: 
    September 2009 – March 2012, Including No-Cost Extension
  • AHRQ Funding Amount: 
    $496,788
  • PDF Version: 
    (PDF, 259.86 KB)


Target Population: Adults

Summary: The office visit remains a cornerstone of primary care delivery and is the major venue where health care information is transferred from provider to patient. Yet studies have shown that by the time the patient leaves a facility, he or she may forget as much as 50 percent of the information relayed during the visit, which can negatively affect a patient’s care.

Electronic medical records (EMRs) offer a new method of providing patients with information about their clinical visits through personalized, patient-specific handouts that summarize the topics and recommendations covered during the visit. These after-visit summaries (AVS) have the potential to improve a patient’s retention of information that is needed for adherence to treatment plans and followup instructions. AVS can also facilitate the transfer of information between health settings; however, the content and formatting of the AVS that will optimize patients’ information retention and satisfaction is not known.

This project is employing qualitative data collection methods, including interviews and focus groups, to gather patient and physician input into AVS development. It is taking place in four clinics that use the Certification Commission for Health Information Technology-certified EpicCare EMR, and that serve an ethnically and socioeconomically diverse patient population. The investigating team includes experts in study design and analysis, medical informatics, bilingual and bicultural patient information transfer, qualitative analysis, and clinical practice-based research.

By the end of the project, the study team plans to develop a prototype of the implementation document and guide, which will be evaluated by at least two practice sites that are interested in health information technology (IT) applications but not involved in the study. These sites will provide comment on the potential usefulness of the document in an implementation effort, and the study team will revise the document as warranted. The team will draft and submit a manuscript about the research effort to a peer-reviewed journal, as well as propose processes for active dissemination, including publicity through primary care organizations, academic primary care departments, practice associations, and various organizations active in health IT development. Post downloadable technical findings and after-visit summary-related products will be available on the Internet for incorporation by other primary care providers. The team will prepare a final report that includes all study deliverables (e.g., copies of all research instruments), results, conclusions, suggestions for additional research, and actionable lessons learned.

Project Objectives:
  • Ascertain patient attitudes, preferences, and needs regarding the delivery of information at a visit with a primary care physician. (Ongoing)
  • Identify primary physicians’ attitudes about the utility, content, and value of the AVS. (Ongoing)
  • Develop and test three different versions of an AVS. (Achieved)
  • Disseminate the programming instructions needed to deploy an AVS for health care organizations that use the Epic EMR system. (Upcoming)

2010 Activities: The study team conducted a series of interviews with patients of the 12 participating primary care physicians to collect qualitative data regarding their information needs and preferences following office visits with their physicians, as well as their general comprehension of health information conveyed through the AVS received at that particular visit. All interviews were conducted using a general interview guide. Interviews were audio recorded, transcribed, and entered into NVivo 8 software, a qualitative data analysis software which helps organize and analyze non-numerical data. Forty-eight interviews (18 of these in Spanish) were completed, and the themes and responses were tabulated.

In 12 one-on-one interviews, following techniques outlined by other researchers for physician interviews, study staff asked physicians about the educational domains surrounding the AVS, as well as the work-flow issues involved with generating them. A physician interview guide was used to engage physicians in discussions of their experience and recommendations for an AVS. The interviewer recorded the sessions and the team encoded the audio using NVivo 8 software. The software allowed researchers to track the major themes and content domains that the interviews captured. Data were summarized using both narrative explanations and direct quotes to emphasize key points.

Program code for the AVS versions was developed and delivered to the participating practice’s IT team. Some internal troubleshooting was necessary because the code was from a slightly different version of Epic; however the IT team was able to address these issues by producing the three versions of the AVS. The three versions were tested and additional modifications were made.

Finally, the research assistants began the process of data collection at the Baylor Family Medicine clinic. Clinic workflow was analyzed and modified to conduct the study and minimize any effect on the workflow for patient care. The team has approval for the draft Patient AVS satisfaction form. The main domains included for evaluation were: content, format, and utility. Questions are formatted in Likert scale format.

Preliminary Impact and Findings: The project has no findings to date.

Strategic Goal: Develop and disseminate health IT evidence and evidence-based tools to support patient-centered care, the coordination of care across transitions in care settings, and the use of electronic exchange of health information to improve quality of care.

Business Goal: Knowledge Creation

This project does not have any related resource.
This project does not have any related survey.
This project does not have any related project spotlight.
This project does not have any related survey.
This project does not have any related story.
This project does not have any related emerging lesson.