Findings From The Evaluation of

E-Prescribing Pilot Sites

 

 

AHRQ Publication No.  07-0047-EF

April 2007

Agency for Healthcare Research and Quality and Centers for Medicare & Medicaid Services logos. 

 

 

 

 

 


Prepared for:

Agency for Healthcare Research and Quality

U.S. Department of Health and Human Services

540 Gaither Road

Rockville, MD 20850

www.ahrq.gov

 

Prepared by:

The National Opinion Research Center (NORC) at the University Of Chicago

 

Disclaimer:
The authors of this report are responsible for its content.  Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services. Identifiable information on which this report, presentation, or other form of disclosure is based is protected by federal law, section 924(c) of the Public Health Service Act, 42 u.s.c. 299c-3(c).  Any confidential identifiable information in this report or presentation that is knowingly disclosed is disclosed solely for the purpose for which it was provided.  No identifiable information about any individuals or entities supplying the information or described in it will be knowingly used except in accordance with their prior consent.

Table of Contents ii

EXECUTIVE SUMMARY. v

Pilot Site Characteristics vi

Methods Used for the Current Evaluation. vi

Findings from Standards Testing vii

Findings from Outcomes Studies ix

SECTION I:  INTRODUCTION.. 1

A Recent History of the Implementation of E-prescribing 2

The Need for Standards 4

Proposed Standards 5

Pilot Sites 7

Objectives of the Evaluation. 7

SECTION II.  PROPOSED STANDARDS. 8

Summary of the Initial Standards 8

Medication History 8

Formulary and Benefits 9

Prescription Fill Status Notification. 9

Prior Authorization. 10

Structured & Codified SIG. 11

RxNorm. 12

How the Standards are Intended to Work Together 12

SECTION III:  OVERVIEW OF CMS/AHRQ PILOT SITES. 15

Overview of the CMS/AHRQ Pilot Site Portfolio 15

Description of CMS/AHRQ Pilot Sites 17

RAND Corporation. 17

Brigham and Woman’s Hospital 17

Achieve Healthcare Information Technologies, LP. 18

Ohio KePRO/ UHMP. 18

SureScripts, LLC. 19

SECTION IV:  EVALUATION METHODS. 20

Research Questions 20

Data Collection Activities 21

Data Analysis 22

Pilot Site Characteristics 23

Pilot Site Evaluation Findings 23

Assessment of Pilot Site Testing and Evaluation Methods 26

SECTION V:   EVALUATION FINDINGS:  INITIAL STANDARDS. 27

Pilot Site Coverage of the Standards 27

Results of Standards Testing 28

Formulary & Benefits 29

Medication History 32

Fill Status Notification. 35

Prior Authorization. 39

Structured and Codified SIG. 44

RxNorm. 46

NCDPD SCRIPT Standard v8.1. 50

SECTION VI:  EVALUATION FINDINGS - OTHER OUTCOMES. 51

E-prescribing Outcomes 52

Prescriber Uptake and Satisfaction. 52

Pharmacy Data Analysis 61

Changes in Number of Callbacks from Pharmacy to Prescribers 68

Patient Satisfaction. 72

Use of Medication History Functions 73

Changes in Prescription Renewal and New Prescription Rates 81

Inappropriate Prescribing Rates 83

Medication Errors, Adverse Drug Events, Hospitalizations and ED Visit Rates 86

Use of On-formulary Medications and Generics 93

Conclusions 97

Change in Fill Status Rates 98

Improved Security and Reliability of Prescriptions 102

SECTION VII: CONCLUSIONS AND RECOMMENDATIONS. 105

Standards Testing 105

Outcomes and Impacts 107

Appendix A:  Site Visit Protocol 109

GLOSSARY. 110

REFERENCES. 115

 

List of Exhibits

Exhibit 1.     Description of Initial and Foundation Standards and Focus of their Testing 6

Exhibit 2.     List of E-prescribing Pilot Sites 7

Exhibit 3.     Information Flow in the Medication History Standard. 8

Exhibit 4.     Information Flow in the Formulary and Benefits Standard. 9

Exhibit 5.     Information Flow in the Prescription Fill Status Standard. 10

Exhibit 6.     Information Flow in the Prior Authorization Standard. 11

Exhibit 7.     Foundation and Initial Standards Required for the Transmission of Electronic Prescriptions, by Step in the Medication Management Process 12

Exhibit 8.     Standard for Interoperable Health Care 13

Exhibit 9.     Settings and Stakeholders Included in Pilot Sites 15

Exhibit 10.       Description of E-prescribing Pilot Sites 16

Exhibit 11.       Description of Data Sources Used to Collect Both Primary and Secondary Evaluation Data. 21

Exhibit 12.       Overview of Evaluation Framework. 23

Exhibit 13.       Summary of Key Factors and Measures/Data Elements Required by the RFA and Used by the Evaluation Team to Inform the Recommendations to CMS. 25

Exhibit 14.       Summary of Standards Tested and Methodology by Pilot Site 28

Exhibit 15.       E-prescribing Outcomes Tested by Pilot Sites 51

Exhibit 16.       Achieve Orders Entered by Staff Type 52

Exhibit 17.       Differences in the Cultures of Study and Control Group Practices 55

Exhibit 18.       Reasons Given for Continuing to Use Paper for Prescribing (RAND) 55

Exhibit 19.       E-prescribing Participation by Month in the Horizon BCBSNJ E-Prescribe Program.. 56

Exhibit 20.       How Prescription Renewal Requests are Received. 63

Exhibit 21.       Reasons to Pull Patient’s Medical Chart 64

Exhibit 22.       Phoned-in Renewal Requests 65

Exhibit 23.       RAND Model of E-prescribing Work Process 66

Exhibit 24.       Least-Used Features of E-prescribing (SureScripts) 67

Exhibit 25.       Number of Incoming and Outgoing Calls per Physician per Day 70

Exhibit 26.       Differences in Time Spent on Phone 71

Exhibit 27.       Differences in the Source or Destination of Phone Calls 71

Exhibit 28.       Differences in Charts Requested to Process Prescription-Related Phone Calls 72

Exhibit 29.       Use of Medication History, by Month (Ohio KePRO/UHMP) 74

Exhibit 30.       Structured Phone Interviews with 3 Internists Regarding Medication History Test 75

Exhibit 31.       Perception on the Benefits of Medication History Function (RAND) 76

Exhibit 32.       SureScripts Exhibit D16. 77

Exhibit 33.       SureScripts Figure D19. 79

Exhibit 34.       SureScripts Figures 25 and 30. 80

Exhibit 35.       Drug-Drug and High Dose DUR Edits by Practice Type 83

Exhibit 36.       Drug-Drug and High Dose DUR Edits by Practice Type and Specialty 84

Exhibit 37.       The Percentage of E-prescription Transactions for a New Prescription for Patients at Least 65 Years of Age that were Identified as Potentially Inappropriate (Fick et al.) by Vendor and Month. 85

Exhibit 38.       The Percentage of Patients at Least 65 Years of Age that were Identified as Receiving at Least 1 New E-Prescription for a Potentially Inappropriate Medication (Fick et al.) by Vendor and Month. 85

Exhibit 39.       Medication Safety and ADE Tables from Brigham and Women’s Hospital 86

Exhibit 40.       Prescriptions for Review and Dispensing Errors 88

Exhibit 41.       Dispensing Error Sub-Categories Routed vs. Non-Routed. 89

Exhibit 42:       Prescriptions for Review and Dispensing Errors Corrected for Possible Different Prescriptions Presented at the Pharmacy 90

Exhibit 43.       Adverse Drug Events by Practice Type, All Data Sources Combined. 91

Exhibit 44.       ADEs by Practice Type, Concept Data. 91

Exhibit 45.       Differences in Formulary Compliance, 3 Months of Health Plan Data. 93

Exhibit 46.       Differences in Generic Dispensing Rates, 3 Months of Health Plan Data. 94

Exhibit 47.       Single Source Dispensing Rates, 3 Months of Health Plan Data. 94

Exhibit 48.       Cost of Brand and Generic Anticholestorimia Drugs 95

Exhibit 49.       Cost Differences by Practice Type 95

Exhibit 50.       Perceptions of Formulary Data in E-prescribing Software (RAND) 96

Exhibit 51.       SureScripts Physician Formulary and Benefits Survey 97

Exhibit 52.       Interest in Continuing RXFILL by Selected Physician. 98

Exhibit 53.       Interest in Continuing RXFILL by Practice Manager 98

Exhibit 54.       Figures D3 – D6 from SureScripts Report 99

Exhibit 55.       E-prescribing Security and eSignature Infrastructure – Long Term Care Alternate Model 103

 


The current system of prescribing and dispensing medications in the United States has widespread problems with safety and efficiency.  Experts predict that a shift to electronic prescribing (e-prescribing) systems could avoid more than 2 million adverse drug events (ADEs) annually, of which 130,000 are life threatening.   E-prescribing also has enormous potential to create savings in health care costs, both in the treatment of these ADEs and in the workflow of prescribers and pharmacists.  One recent study estimated the potential savings at $27 billion per year in the United States.  

However, adoption of e-prescribing technology remains limited.  One major hurdle to effective implementation of e-prescribing has been the inability of multiple systems to share information effectively.  Lacking a standard format and vocabulary, systems do not always effectively and unequivocally communicate the necessary information among all participants in the transaction. This reduces the effectiveness and attractiveness of using an electronic system. 

Because of e-prescribing’s proven potential to reduce medication errors and the cost of medical care, in the Medicare  Prescription Drug Improvement and Modernization Act (MMA) of 2003, Congress mandated that all plans and pharmacies participating in the new Medicare prescription drug benefit (Part D) support an electronic prescription program.  Although prescribers are not required to use e-prescribing, plans must have a system in place for those who do want to use e-prescribing technology. 

To address the multiple formats and vocabularies that present barriers to implementation, the MMA directed the Secretary of Health and Human Services (HHS) to establish federal standards that all e-prescribers must follow for Part D patients. These standards are published specifications that establish a common language, contain technical specifications, and provide other specific criteria designed to be used consistently as rules or definitions. 

When HHS promulgated rules proposing standards for e-prescribing, the rules identified three well-accepted standards ready for immediate implementation, “foundation” standards, and several other areas in which standards are needed.  In these areas, HHS proposed six “initial” standards for pilot testing. 

HHS made grants to five pilot sites to test the standards.  These pilots were set up to test initial standards and their interoperability with foundation standards as well as clinical and economic outcomes associated with e-prescribing. The Agency for Healthcare Research and Quality (AHRQ) National Resource Center for Health IT (NRC) was then charged with compiling the current report which summarizes and synthesizes findings across these pilot sites with the goal of advising the federal government on standards adoption and disseminating key data on e-prescribing outcomes among the policy community. In the remainder of this Executive Summary we outline pilot characteristics, the methodology used by the NRC in conducting this cross-cutting assessment of pilot findings, and summary-level information findings from both standards and outcomes testing.

Pilot Site Characteristics

Each site selected for the pilot has the potential to produce special information for the government based on the standards they tested, methodologies used, and context in which e-prescribing was implemented or assessed. Key features of each of the pilot sites are described below.

§         RAND focused on New Jersey physicians in an e-prescribing program sponsored by Horizon Blue Cross Blue Shield of New Jersey.  The pilot also included partnerships with Caremark’s mail-order pharmacy and Walgreens’ retail pharmacy, so that the project could include end-to-end testing of the standards.

§         Brigham and Women’s Hospital worked with physicians from the CareGroup Health System in Boston who were already using mature outpatient electronic medical record (EMR) and computerized physician order entry (CPOE) systems.  This enabled them to isolate the effects of the standards on already operational e-prescribing practices.

§         Achieve, the largest information technology vendor for the long term care (LTC) industry, partnered with a nonprofit LTC system in the Midwest that also owns the pharmacies that serve its facilities and RNA, a pharmacy management system software vendor for LTC settings.  This pilot study implemented e-prescribing in facilities that had never used the technology before.

§         University Hospitals Health System and Ohio KePRO, the Quality Improvement Organization in Ohio, teamed to study the implementation of the standards in some of the 300 primary and specialty care physician offices that make up the University Hospitals Medical Practices.  These physicians are generally in small practices of two to three doctors, a very common practice environment.

§         SureScripts is the nation’s largest provider of e-prescribing networking and certification services.  They worked with physician offices in Florida, Massachusetts, Nevada, New Jersey, and Tennessee using a variety of software systems to send prescriptions to an assortment of chain and independent pharmacies.

Pilot sites used a variety of techniques to test standards, including interviews and expert panels, live transactions encompassing an end-to-end prescribing process, and simulation of data transactions in laboratory settings.  This report provides a look at the results of these projects.

Methods Used for the Current Evaluation

In order to gather results surrounding the six initial standards from the pilot sites, the NORC evaluation team visited each pilot site, held structured conference calls, and reviewed written materials from each site.  Written materials included grant proposals, quarterly reports, and final reports.  Because there was great variability among the pilot sites in terms implementation site, technology system vendors, and standards tested, the evaluation team took into consideration the characteristics of the pilot sites, as well as the testing methods they used to test standards.

The evaluation team gauged the strength of each of the pilot site’s research designs and methodology relative to accepted standards in the fields of qualitative and quantitative research.  This exercise allowed the team to reach informed conclusions regarding how each pilot site-level result should be used in developing final recommendation for CMS.

Findings from Standards Testing

Having analyzed the sites’ findings in the context of their characteristics and testing methods, the evaluation team makes the following recommendations on the initial standards:

§         Medication History.  The medication history standard is intended to provide a uniform means for prescribers, dispensers, and payers to communicate about the list of drugs that have been dispensed to a patient.  This standard is relatively mature and widely adopted by the e- prescribing industry.  It has been shown to be useful in preventing medication errors, as well as understanding medication management compliance. 

The evaluation team recommends that this standard is ready for implementation under Part D.  In general, the pilots found that the proposed standard is structured well and is well suited for the exchange of information.  The main challenge will be ensuring that data are collected and reconciled from a large number of sources to ensure that a patient’s medication list is complete.  This is an issue of implementation, not something that can be addressed within the standard itself.

§         Formulary and Benefits.  The formulary and benefits standard is intended to provide prescribers with information about a patient’s drug coverage at the point of care.  Information may include whether drugs are considered to be "on formulary," alternative medications for those drugs not on formulary, rules for prior authorization and step therapy, and the cost to the patient for one drug option versus another.  The goal is to enable the prescriber to take this information into account at the time of prescribing, reducing the amount of back-and-forth communication needed with the pharmacy or the health plan.  This standard is currently being used by some prescribers to obtain formulary and benefit information, with one system integrator quoting a volume of over 3.5 million transactions per month.

The evaluation team recommends that this standard is ready for implementation under Part D. The Formulary and Benefits standard adequately supports the transfer of the intended information.  As with medication history, however, there are important implementation issues.  First, systems must adequately match patients to health plans, or the formulary and benefits data will not be available.  Second, payers vary in the level of information that they provide, and data elements can be difficult to interpret even when they are transmitted accurately.  Finally, to be most useful, this transaction should support real-time changes in a patient’s status as he or she moves through different stages of a benefit (such as the Part D “doughnut hole”). 

§         Prescription Fill Status Notification.  The purpose of the prescription fill status transaction is to notify the prescriber about whether a patient has picked up a prescribed medication at the pharmacy.  This information could enable follow-up with patients who appear to be non-compliant with their doctor’s prescribed course of treatment. 

The evaluation team recommends that this standard is ready for implementation under Part D.  The standard is sufficient to support the activities of a pharmacy sending messages to the prescriber as to the status of a prescription, when the information is available.  However, many pharmacies do not have the ability to track patient pick-up accurately.  In addition, the pilots indicated that there may be little prescriber demand for this capability.

§         Prior Authorization.  Prior authorization is a process by which insurers require patients to receive approval before certain drugs will be covered.  Often, physicians must certify that a patient meets specific, defined criteria for the use of the drug.  The current system requires multiple phone and written contacts between the prescriber, the pharmacist, and the health plan. Electronic prior authorization would create a streamlined process to communicate the need for prior authorization directly to the prescriber, and allow the prescriber to send the needed information along with the prescription. 

The evaluation team does not recommend the implementation of this standard in its current state.  Because health plans typically require prior authorization only for a small subset of drugs, the pilot sites had limited live experience with this standard.  The pilot sites examined various approaches to assessing the potential impact of a standardized electronic prior authorization (e-PA) on the prescriber’s workflow, changes in prescribing behaviors and perceptions of access to appropriate medications both in lab environments and live implementations.  Pilot sites identified several issues that would need to be resolved before this standard is recommended. 

§         Structured and Codified SIG.  Patient instructions for taking medications (such as “by mouth, three times a day”) are placed at the end of a prescription. These are called the signatura, commonly abbreviated SIG.  Currently, there is no standardized format or vocabulary for SIGs, leaving room for misinterpretation and error. Standardizing and codifying SIGS would enhance patient safety.

The evaluation team does not recommend the implementation of this standard in its current state.  Pilot tests found that the Structured and Codified SIG format needs additional work with reference to field definitions and examples, field naming conventions and clarifications of field use. With additional development, the standard may provide a controlled vocabulary that reflects prescriber thinking, offers structure and simplicity, and improves communications between prescribers and pharmacies.

§         RxNorm.  There are currently multiple databases of drug names, forms, and dosages.  Each may use slightly different versions of these data elements, requiring an individual at the pharmacy to make a manual match if a prescription is communicated using information from a different database.  RxNorm, a system designed for federal government entities, would provide standards for the name, dose, and form of available drugs.  

The evaluation team does not recommend implementation of versions 8/2/06 and 12/21/06 of this standard.  RxNorm has the potential to create efficiencies in many e-prescribing functions. However, the dictionary standard requires further evaluation and refinement before it can be deployed in a live setting.

The long term care project also had substantial findings for one of the foundation standards. This project found that one of the foundation standards (SCRIPT v8.1) needed revision to accommodate their prescribing workflows.  This site is working with the National Council for Prescription Drug Programs (NCPDP), the organization that developed the standard, to make needed modifications.  These include the need to update prescriptions without having to create a new order, the ability to send a refill from the facility to the pharmacy without the physician’s intervention, and the ability to update patient information outside the context of a prescription.

Findings from Outcomes Studies

In addition to testing the functionality of e-prescribing standards, pilot sites tracked various outcomes of e-prescribing in their pilots.  The following observations were made by the evaluation team:

§         Prescriber uptake and satisfaction. Adoption and retention of e-prescribing among providers was generally good. In order to facilitate prescriber adoption, the evaluation team recommends institutions implementing e-prescribing take into account the role of their organizational culture and prepare for possible “surrogate prescribing” (see below).

§         Prescriber and pharmacy workflow changes. One finding that was consistent across all sites was that prescribers’ staff played a much more important role in the e-prescribing process than most pilot sites had anticipated.  The evaluation team recommends that future e-prescribing efforts take the role of these staff, or “surrogate prescribers” into account in their planning. Another finding was that e-prescribing almost never replaced the need for paper-based prescribing, leading to highly variable use of e-prescribing features. In addition, implementation of e-prescribing can create dramatic “paradigm shifts” in pharmacy workflow. Pharmacies implementing e-prescribing, therefore, must allocate sufficient resources to deal with substantial change management. Finally, preliminary findings suggest that e-prescribing tools may decrease reliance on verbal orders and generate certain efficiencies for small physician offices. Proof of such efficiencies is still relatively preliminary, however.

§         Changes in number of callbacks from pharmacy to prescribers. Findings reported by some pilots suggest that e-prescribing reduces the number of phone time for physician practices while potentially decreasing efficiency on the pharmacy through an increase in the number of callbacks required to complete a prescription.  Yet other pilots found a decrease in callbacks related specifically to drug coverage issues.  Given these inconsistencies, the evaluation team recommends that further study is required to acquire a more complete understanding of this potentially “cost-shifting” phenomenon.

§         Patient Satisfaction. According to surveys from one pilot site, most patients are satisfied with e-prescribing. Future studies should investigate further into patient perspectives to see what may cause dissatisfaction.

§         Use of Medication History functions. Overall, the pilots’ findings demonstrated poor adoption of this functionality. We recommend further research to determine better ways for displaying and maintaining up-to-date medication histories to providers.

§         Changes in prescription renewal and new prescription rates. The long term care site reported a reduction in new prescription rates, indicating the possibility that e-prescribing may reduce the tendency for such patients to accumulate unnecessary active medications.

§         Inappropriate prescribing rates. The study period was too brief to make a measurable difference in the number of inappropriately prescribed medications.

§         Medication errors, Adverse Drug Events, Hospitalizations and ED visit rates. The data on medication errors and ADEs is not conclusive and is in a preliminary state. The pilots will proceed with additional analysis to determine more precisely the impact of e-prescribing on patient safety.

§         Use of on-formulary medications and generics. Clinicians surveyed by the pilots were concerned about the accuracy of formulary information provided by e-prescribing systems. Further studies will need to assess the perceived and actual quality of this information. In addition, generic prescribing that automatically allow for generic substitution may increase the rate of generic prescribing.

§         Change in fill status rates. Fill status use was extremely limited due to the difficult implementation of this standard.

§         Improved security and reliability of prescriptions. Only one of the sites investigated this issue; however, the security architecture they developed shows that the industry is taking important steps towards implementing systems that are secure and reliable. Future studies should test e-prescribing to ensure it meets security standards.


Efforts to modernize the American health care system have accelerated over the last five years, due in large part to several landmark studies revealing the startling toll of medical and medication errors. In 1999, the Institute of Medicine estimated that as many as 7,000 people die each year from medication errors alone, accounting for 1 out of 131 ambulatory deaths.[i]  In hospitals, the average patient is subjected to at least one medication error per day.[ii]  A recent study by the Center for Information Technology Leadership showed that 8.8 million Adverse Drug Events (ADEs) occur each year in ambulatory care. Also from this study came the even more troubling statistic that one quarter, or 3 million, of these errors were preventable.[iii] 

Medication-related errors cost money in addition to costing lives. Preventable ADEs occurring in hospitals cost the American health care system $3.5 billion per year, [iv] while those in ambulatory settings amount to upwards of $887 million.[v] In addition, a paper-based system may create costs due to inefficient workflows.[vi]  For example, illegible handwriting is a widespread problem that not only causes errors but also uses staff time to determine the physician’s intent.  For this and other reasons, almost 30% of prescriptions require pharmacy call backs, resulting in 900 million prescription-related telephone calls annually.[vii]

In order to address these concerns with safety and efficiency, scholars, health experts, and industry leaders have supported the switch from a paper to an electronic system of prescribing. E-prescribing is considered safer[viii] because it ensures that meaningful and relevant data are communicated to the people who need it, when they need it.[ix],[x]  For pharmacists, e-prescribing can better communicate the prescriber’s intent, eliminating issues with illegible handwriting or confusing directions.  For prescribers, e-prescribing systems can include clinical decision support (CDS) systems that check the patient’s medical history and provide information about possible allergies, drug-drug interactions and dosing issues. [xi]  Systems can also check a patient’s insurance coverage, notifying the prescriber when a drug is not covered or requires prior authorization from the insurer. 

As a result of these improvements over a paper-based system, experts predict e-prescribing systems can avoid more than 2 million ADEs annually, of which 130,000 are life-threatening.[xii] In addition to reducing the medical spending associated with treating these ADEs, such computer systems could generate other savings.  For example, e-prescribing has the potential to allow providers to make more informed decisions about clinically appropriate and cost-effective medications.[xiii] According to the Center for Information Technology Leadership, an additional cost savings of $2.7 billion would result from e-prescribing’s ability to reduce clinicians’ phone time.[xiv]  The e-Health Initiative recently estimated that widespread adoption of e-prescribing could save the United States healthcare system $27 billion per year.[xv]

Because of e-prescribing’s proven potential to reduce medication errors and the cost of medical care, Congress mandated in the Medicare Prescription Drug Improvement and Modernization Act (MMA) of 2003 that all plans participating in the new Medicare prescription drug benefit (Part D) support an e-prescribing program.  Although prescribers are not required to participate, the plans must have a system in place for those who do want to use e-prescribing technology.  In its requirement that all Part D plans support e-prescribing, the MMA also required that all such programs follow federal standards promulgated by the Secretary of Health and Human Services.  This report analyzes the readiness and potential impact of several proposed standards.

A Recent History of the Implementation of E-prescribing

Over the last four years, several private organizations, states, and regional collaboratives have sought to implement and evaluate the effectiveness of integrated e-prescribing systems:

§         In 2002, Tufts Health Plan of Massachusetts and Advance PCS (now Caremark) conducted a year-long pilot study of integrated e-prescribing. The study involved over 100 clinicians, and found that e-prescribing had positive effects on patient safety, cost, pharmacy and prescriber efficiency, and user satisfaction.[xvi]

§         In 2003, Blue Cross Blue Shield of Massachusetts, in conjunction with various industry partners, launched the eRx Collaborative. The Collaborative’s goal was to deploy e-prescribing systems in the offices of 3,400 Massachusetts physicians.[xvii] After two years, the program had allowed for over three million prescriptions to be transmitted electronically. With regards to safety, by 2005 more than 5,500 prescriptions per month were being changed as a result of warning messages built into the system.[xviii]

§         In 2003, the Rhode Island Quality Institute started a program to implement statewide e-prescribing throughout the state of Rhode Island. Aided by the state’s appropriation of $20 million towards Health Information Exchange, as well as underwriting from e-prescribing network SureScripts and vendor LighthouseMD, the public, private, and academic partnership was met with great success. In 2006, Rhode Island was ranked first in the country in e-prescribing.[xix]

§         In 2003, the Massachusetts Medical Society developed a strategic plan for implementing interoperable e-prescribing throughout the state of Massachusetts. Later, in 2005, the society subsidized e-prescribing for its members. Massachusetts was recently ranked third in e-prescribing, according to pharmacy groups.[xx]-[xxi]

§         In 2004, Wellpoint, Inc. of Indiana introduced the Physician Quality and Technology Initiative (PQTI). The program gave e-prescribing and administrative software to over 19,000 physicians in HMOs and PPOs in California, Georgia, Missouri and Wisconsin. Although the majority of physicians enrolled in this program use only the administrative software program, the preliminary results surrounding e-prescribing are encouraging. As of July 2005, over 90,000 prescriptions had been written electronically, and offices using the e-prescribing utility were spending 75% less time on administrative work.[xxii]

§         In 2004, CareFirst Blue Cross Blue Shield, Maryland’s largest insurer, decided to provide DrFirst’s Rcopia to 500 physicians.  CareFirst and DrFirst also gave wireless handheld devices developed by spring and palmOne Inc. to participating physicians.[xxiii] After the program’s first year, it was estimated that over $1.3 million in cost savings could be directly attributed to reduced prescribing errors.[xxiv]

§         In 2005, Blue Cross Blue Shield of Michigan and the big three American automakers joined forces with local pharmacy benefit managers and HMOs to launch the Southeast Michigan e-prescribing Initiative (SEMI). Over 17,000 physicians had the opportunity to participate in the program, which successfully developed an interoperable e-prescribing system throughout the region.[xxv] Between August 2005 and April 2006, Henry Ford Medical Group reported over 588,000 electronic scripts in total, with 70,000 prescriptions canceled due to drug-drug interaction warnings and over 4,500 prescriptions canceled due to allergy alerts.[xxvi] In addition, over the course of the entire project, researchers found a net reduction in pharmacist initiated calls, as well as a savings in physician telephone time. Together, these efficiencies could save an estimate $2.9 billion per year. Other findings included increased formulary compliance rates and increased generic dispensing for physicians who used e-prescribing.[xxvii]

§         In 2005, the state of Delaware launched a pilot e-prescribing program with 100 physicians, using the system Rcopia, developed by DrFirst. Results from the pilot study will be published in late 2006.[xxviii] [xxix] As of June 2006, more than 75,426 prescriptions had been written electronically.[xxx]

§         In 2005, Sierra Health Services, it subsidiaries Health Plan of Nevada and Southwest Medical Associates (SMA), and the Clark County Medical Society of Nevada funded a program to provide all 5,000 physicians in the state with e-prescribing software. All Nevada physicians were eligible to receive a free license for Allscripts’ Touchworks Rx+ application.[xxxi] Since the implementation of the application, use of generic drugs among patients increased from 59% of prescriptions written to 65%; this increase translates into an annual cost savings of $5 million.[xxxii] In addition, callbacks from pharmacies declined and patient satisfaction increased.[xxxiii]

§         In January 2006, Blue Cross Blue Shield of North Carolina started “ePrescribe,” an initiative that provided funding for e-prescribing software, hardware, and support for 1,000 physicians in North Carolina.[xxxiv]

§         In May 2006, L.A. Care, the largest Medicaid HMO in the country, began an e-prescribing pilot program. The Los Angeles-based health care payer purchased hardware and software from Zix Corporation and provided it free of charge to participating physicians. The initial pilot study involved 50 physicians, but may ultimately include as many as 100.[xxxv]

§         In October 2006, New Hampshire Governor John Lynch announced the goal of having all doctors in his state prescribing electronically by 2008.  According to experts, New Hampshire has a good chance of achieving this goal, due in part to its small size, as well as the fact that in 2006, New Hampshire already possessed a fairly advanced health information system—nearly 75% of the state’s primary care physicians used EHRs, and up to 80% of the state’s pharmacies had e-prescribing capabilities.[xxxvi]

§         In January 2007, a coalition of technology companies and healthcare organizations calling themselves the National e-prescribing Patient Safety Initiative announced a program aimed at providing free e-prescribing to every physician in America. The coalition is led by Allscripts and Dell, and includes Aetna, Cisco Systems, Fujitsu Computers of America, Google, Microsoft, Sprint Nextel, SureScripts, Wellpoint, and Wolters Kluwer Health. Through web-based software available for free at www.natinoalerx.com, the coalition hopes to appeal to small practice physicians who otherwise would not wish to purchase an e-prescribing system.[xxxvii]

§         In January 2007, TennCare, Tennessee’s Medicaid program, received a $674,200 HHS grant to launch an electronic prescription system linking rural doctors and pharmacies. The main goals of the program are to reduce pharmacy costs and increase patient safety by providing physicians in rural communities with computers to transmit prescriptions to local pharmacies.[xxxviii]

§         In February 2007, five of Florida’s largest health plans announced a joint effort to encourage the State’s physicians to use e-prescribing. The collaborative, named “e-Prescribe Florida” involves Blue Cross Blue Shield of Florida, Av-Med, Cigna, Humana, and UnitedHealthcare.[xxxix]

 

The momentum is building in the industry to drive adoption of e-prescribing as a result of the MMA legislation and the desire to create efficiencies, reduce costs and improve patient safety.

The Need for Standards

In spite of the progress made by numerous projects, the adoption of e-prescribing technology remains limited. According to industry surveys, only 5% to 18% of doctors use any form of e-prescribing.[xl]  Fewer than 3 percent of all prescriptions are written with integrated e-prescribing systems that realize the most significant benefits of e-prescribing. [xli]  Ideally, these integrated systems communicate clearly, securely, and easily across key steps in the drug delivery chain – from accessing information vital to choosing medications, to ordering medications, to dispensing drugs, to payment.[xlii].[xliii]

Prescribers, pharmacy dispensers, software vendors, insurers, and patients must work together in order for the integrated e-prescribing system to become a reality.  In order to share critical information across various health care settings, systems must be able to interoperate with one another.  The inability for multiple systems to share information with a standard format and vocabulary has been a hurdle to effective implementation of e-prescribing.[xliv] The few data standards that are available often are not published with sufficient precision to be implemented in a way that can be constructed as a true “standard.”[xlv] As a result, participants in the e-prescribing chain may have to use more time-consuming “workarounds” to transfer information and accomplish other functions of e-prescribing.[xlvi]  Solutions like the manual re-entry of data lessen the potential safety and economic benefits of using e-prescribing in the first place.

To address this situation, the National Committee on Vital and Health Statistics (NCVHS) was called upon by the MMA to develop recommendations for uniform standards to enable e-prescribing in ambulatory care.  A standard is a published specification that establishes a common language, and contains a technical specification or other specific criteria and is designed to be used consistently, as a rule, a guideline, or a definition.  The specific data and supportive process must “match” at both the source and destination computer systems -- which is only achievable with adherence to using the same standards.  Thus, standards are the fundamental building blocks essential for the widespread adoption of e-prescribing as well as other health information technologies (HIT). 

NCVHS identified and evaluated three types of e-prescribing standards as necessary requirements to support e-prescribing.  Message format standards provide communication protocols and data content requirements (including those that support medication decision making).  Terminologies ensure data comparability and interoperability.  Identifiers for all relevant entities within the e-prescribing process allow for clearer tracking and communication.   

The MMA requires that all Part D plans follow federal standards promulgated by the Secretary of Health and Human Services when they implement e-prescribing systems.  Compliance with the standards is also required for prescribers and dispensers that choose to send or receive prescription-related information electronically for covered Part D drugs for Part D eligible individuals.[xlvii]  Although prescribers and pharmacies will not be required to use these same standards for their other patients, the adoption of standards for Part D will pave the way for more integrated systems across the board.

Proposed Standards

When HHS started to circulate rules proposing standards for e-prescribing, the rules differentiated between those that were available for immediate adoption and those that needed further testing (See 70 FR 6256 (February 4, 2005) and 70 FR 67573 (November 7, 2005), to be codified at 42 CFR 423). The criteria for immediate adoption included: 

§         Approval by an ANSI-accredited SDO to assure consideration of industry requirements.

§         Implementations among multiple partners to assure interoperability.

§         Recognition by key stakeholders to assure industry recognition of a single standard.

 

Three standards met these criteria, and have been adopted as “foundation” standards for the new Part D requirement (see Exhibit 1).

In addition, HHS identified several areas in which standards are needed, but no single standard has been widely adopted.  The MMA called for the Secretary of Health and Human Services to propose such standards and to sponsor pilot sites to evaluate them.[xlviii] The Secretary selected six such new or emerging standards, called “initial” standards (see Exhibit 1).

Exhibit 1.        Description of Initial and Foundation Standards and Focus of their Testing

Name

Standard Description

Testing Requirements

Initial Standards

NCPCP Formulary and Benefits Standard Version 1.0

Displays the formulary status and alternative drugs as well as co-pays and other status information. NCPDP has developed a standard using RxHub protocol.

Determine if it should be adopted as a standard

NCPDP SCRIPT Medication History

Includes the status, provider, patient, coordination of benefit, repeatable drug request, and response segments of SCRIPT.

Determine readiness of the NCPDP's standard medication history message

NCPDP SCRIPT:

Fill Status Notification

 

Informs when Rx filled, not filled, or partially filled. Includes provider,
patient, and drug segments of SCRIPT message. Not yet generally used.

Assess the business value and clinical utility

Structured and Codified SIG

Indication, dose, dose calculation, dose restriction, route, frequency,
interval, site, administration time and duration, stop

Test structured and codified SIGs (patient instructions) developed through standards development organization efforts

RxNorm-Clinical drug terminology (Versions 8/2/2006 and 12/21/2006)

A clinical drug nomenclature that provides standard names for clinical drugs and for dose forms as administered. It also provides links from clinical drugs to their active ingredients, drug components, and most related brand names.

Determine whether RxNorm terminology translates to NDC for new prescriptions, renewals and changes

Prior authorization messages

Requires header information, requester, subscriber, utilization management, and other relevant information for prior authorization requests

 

Determine functionality of new versions of the ASC X12N 275/278. (with HL7 attachment)

Foundation Standards

NCPDP Telecommunications

HIPAA standard for eligibility communications between retail pharmacy dispensers and payers/PBMs.

Determine interoperability with new standards.

ASC X12N-270/271

HIPAA standard for eligibility and benefits communications between dentists, professionals, institutions, and health plans.

Determine interoperability with new standards.

NCPDP SCRIPT Standard Version 5, Release 0:

           New

           Change

           Renewal

           Cancellation

Provides for the exchange of new prescriptions, changes, renewals, and cancellation notifications. Each function has varying degrees of industry experience.

Determine interoperability with new standards.

Long Description: The exhibit list the six new or emerging standards, called “initial” standards which include  NCPDP SCRIPT, Medication History, NCPDP SCRIPT: Fill Status Notification, Structured and Codified SIG, RxNorm-Clinical drug terminology, and Prior authorization messages.

Pilot Sites

During calendar year 2006, the initial standards were tested in five sites (see Exhibit 2).  The sites included a variety of different settings, including long term care and small and large physician practices, to determine whether the standards are ready for broad adoption.  Sites were asked to determine whether the initial standards allow participants to effectively and unequivocally communicate necessary information between all participants in the transaction, such as the pharmacy, pharmacy benefits manager (PBM), router, plan and prescriber.  They were also asked to explore how the initial standards worked with the more accepted foundation standards.  Pilot sites also tracked outcomes in their projects that could be attributed more generally to the use of e-prescribing, such as a reduction in medical errors.

Exhibit 2.     List of E-prescribing Pilot Sites

Organization Name

Project Name

RAND Corporation

Test of Medicare’s initial e-prescribing standards in the New Jersey e-prescribing Action Coalition

Brigham and Woman’s Hospital

E-prescribing using a Community Utility: The e-prescribing Gateway

Achieve Healthcare Information Technologies, LP (LTC)

LTC e-prescribing Standards Pilot Study

Ohio KePRO/UHMP

A Practice-Based Pilot Test of Emerging e-prescribing Standards

SureScripts, LLC

Maximizing the Effectiveness of e-prescribing Between Physicians and Community Pharmacies

                                                                                                                                                                 

Long Description: Provides a list of the five pilot sites which include:  RAND Corporation, Brigham and Woman’s Hospital, Achieve Healthcare Information Technologies, LP (LTC), Ohio KePRO/UHMP, and SureScripts, LLC.

The Secretary of Health and Human Services will consider the results of these pilot sites prior to promulgation of a final set of uniform standards, which are due by April, 2008. The use of this pilot test data and findings will allow for a set of standards that benefits all interested stakeholders: clinicians, medication dispensers, Part D Sponsors, and most importantly, patients and their families.  This report provides a look at the results of the five pilot sites.

Objectives of the Evaluation

The evaluation team was charged with providing the information needed to make informed decisions regarding the initial standards’ functionality, interoperability with foundation standards, and impact on workflow, clinical, and other outcomes.  Specifically, the primary objectives of the evaluation are to: 

1)      systematically collect and interpret the evidence (e.g., testing methods and findings) reported by e-prescribing pilot sites;

2)      determine the initial standards that are reported to be functional (and those which are not),

3)      document the benefits, challenges, and technical considerations for mass implementation of the initial standards in different settings, and

The following sections describe the proposed standards (Section II) and the pilot sites (Section III) in more detail.  Section IV then describes the approach taken by the evaluation team to critically collect data from pilot sites and the methods used to evaluated their findings and assess the strength of their claims.  In Section V, are detailed descriptions of the pilots' experiences with the initial standards.  Section VI presents the findings to date on the other outcomes of e-prescribing in these pilots. The final section of this report synthesizes the conclusions and recommendations to be drawn from the analysis of findings reported by the pilot sites.


The National Committee on Vital and Health Statistics (NCVHS) developed recommendations for uniform standards to enable e-prescribing in ambulatory care.  The Secretary of Health and Human Services then promulgated regulations that proposed six initial standards to be pilot tested in 2006.  This section provides more information on the initial standards and how they are intended to work together.

Summary of the Initial Standards

Medication History

Medication history is an important data element that can help physicians and pharmacists avoid drug-drug interactions and other adverse drug events.  There are several potential sources for medication history:  payers/pharmacy benefit managers (PBMs) can provide paid claims, pharmacies can provide dispensed medications, prescribers can provide medical record information, and patients themselves can self report. 

The NCPDP SCRIPT Medication History standard is a request transaction with a corresponding reply transaction (see Exhibit 3). The requesting entity generates a patient specific Medication History request transaction and sends that request providing sufficient information to uniquely identify the patient. The request is then routed to the appropriate entity for processing. The processing entity must return the medication list (prescriptions) that fill the request criteria in the order of the most recent date filled first. Patient consent must be evaluated prior to completing requests for accurate reporting.

Long Description: The exhibit illustrates the NCPDP SCRIPT Medication History standard’s information flow, which includes a request transaction with a corresponding reply transaction.  The requesting entity generates a patient specific Medication History request transaction and sends that request providing sufficient information to uniquely identify the patient. The request is then routed to the appropriate entity for processing.

Short Description: Information Flow in the Medication History Standard
Exhibit 3.     Information Flow in the Medication History Standard

Formulary and Benefits

The formulary and benefits transaction (see Exhibit 4) allows for standard means for pharmacy benefit payers (including health plans and PBMs) to communicate formulary and benefit information to prescribers via technology vendor systems. It enables the prescriber to consider the following kinds of information during the prescribing process: 1)  information about which drugs are considered to be "on formulary," and alternative medications for those drugs not on formulary 2) limitations that may impact whether the patient's benefit will cover a drug being considered (such as age limits, gender limits, prior authorization, step therapy rules, benefit-specific coverage exclusions, etc), and 3) the cost to the patient for one drug option versus another. 

This standard is being used by prescribers to obtain formulary and benefit information today, with one system integrator quoting a volume of over 3.5 million transactions per month. 

 

Exhibit 4.     Information Flow in the Formulary and Benefits Standard

 

Text Box: PrescriberText Box: Insurer or PBM

 

Long Description: The exhibit illustrates the information flow for the formulary and benefits transaction which allows for standard means for pharmacy benefit payers (including health plans and PBMs) to communicate formulary and benefit information to prescribers via technology vendor systems.

Prescription Fill Status Notification

The purpose of the prescription fill status transaction is to notify the prescriber about the status of a new or refill prescription.  This information could enable providers to follow-up with patients who appear to be non-compliant with their doctor’s prescribed course of treatment.

Exhibit 5 illustrates the information flow involved in the Prescription Fill Status standard. The fill status transaction is originated by the pharmacy in three scenarios: 1) notification of a dispensed prescription (the patient picked up the medication), 2) notification of a partially dispensed prescription (patient picked up part of the medication), and 3) notification that the prescription was not dispensed (patient did not pick up the medication).  The RXFILL (dispensed) message should not be triggered simply by label printing or adjudication in the pharmacy, but through a specific affirmative indication that the medication was actually picked up by the patient.

Exhibit 5.     Information Flow in the Prescription Fill Status Standard

Long Description: The exhibit illustrates the information flow involved in the Prescription Fill Status standard. The fill status transaction is originated by the pharmacy in three scenarios: 1) notification of a dispensed prescription (the patient picked up the medication), 2) notification of a partially dispensed prescription (patient picked up part of the medication), and 3) notification that the prescription was not dispensed (patient did not pick up the medication).

Prior Authorization

Prior authorization (PA) is a process by which insurers require patients to receive approval before certain drugs will be covered.  Often, physicians must certify that a patient meets specific, defined criteria for the use of the drug.  The current system is an iterative, multi-step process.  Online edits in the point-of-sale system typically alert the pharmacist that a particular drug requires PA.  The pharmacist must then communicate this requirement to the physician, and the physician must provide the needed information.  The electronic prior authorization standard would create a streamlined process to communicate the need for prior authorization directly to the prescriber, and allow the prescriber to respond directly with the needed information at the time of writing the prescription.  This process would eliminate several administrative steps and could help patients receive their medications on a timelier basis.  Exhibit 6 shows the information flow process involved in PA request.

 

Exhibit 6.     Information Flow in the Prior Authorization Standard

 

Long Description: The exhibit shows the information flow process involved in PA request, which includes sets of tasks processes by both payers and providers.  There were two models of electronic PA to consider – solicited and unsolicited.  Under the solicited model, the prescriber requests questions from the health plan or PBM.  Under the unsolicited model, the questions and criteria have been distributed in batch to the point-of-care software systems and the clinician has all the questions needed for a particular drug before beginning the PA request.

There were two models of electronic PA to consider – solicited and unsolicited.  Under the solicited model, the prescriber requests questions from the health plan or PBM.  Under the unsolicited model, the questions and criteria have been distributed in batch to the point-of-care software systems and the clinician has all the questions needed for a particular drug before beginning the PA request.   All pilot sites selected this unsolicited model.

Structured & Codified SIG

Patient instructions for taking medications (such as “by mouth, three times a day”) are placed at the end of a prescription. These are called the signatura, commonly abbreviated SIG.  Currently, there is no standardized format or code set for transmitting SIGs, leaving room for misinterpretation and error. Thus, standardizing and codifying SIGS will enhance patient safety. The standard breaks down the SIG into components, such as dose, frequency, and maximum dose.  It uses a database of over 1,300 terms that can be used to populate these fields.  In addition, it allows for free text entry to provide additional flexibility.

Compared to other proposed initial standards, the codified SIG is in a more developmental phase. NCPDP, HL7, and others are working on addressing structured SIG components and plan to seek broad industry participation.  At the time of the pilots’ initial start date, the likelihood that the proposed standard would be balloted and adopted by NCPDP was not a near–term prospect.  Thus, this standard was tested in a more theoretical manner in this evaluation.

RxNorm

There are currently multiple databases of drug names, forms, and dosages.  Each may use slightly different versions of these data elements, requiring an individual at the pharmacy to make a manual match if a prescription comes in using information from a different database.  RxNorm, a system designed for federal government entities, would provide standards for the name, dose, and form of available drugs.   RxNorm also provides links from clinical drugs, both branded and generic, to their active ingredients, drug components, related brand names, and NDCs (National Drug Codes). By providing links between these vocabularies, RxNorm can reconcile messages between systems that use different software and vocabulary. 

How the Standards are Intended to Work Together

The proposed standards affect each of the five medication management steps/activities in the process model of medication management (see Exhibit 7).  The first step, “prescribe,” requires the active involvement of a prescribing clinician, as it requires that the clinician assess the patient’s need for prescription medications.  This step is informed by drug information, patient data, and drug formulary restrictions, which may be available from print or electronic resources.  Initial standards that may be involved in this activity include prior authorization, medication history, formulary and benefits, RxNorm, and medication fill status.  Step 2, “transmit,” is where the actual prescription is delivered for fulfillment.  It involves both foundation and initial standards such as: Prior Authorization, structured and codified sig, formulary and benefits, NCPDP telecom, and eligibility and verification. 

Exhibit 7.     Foundation and Initial Standards Required for the Transmission of Electronic Prescriptions, by Step in the Medication Management Process

Source:  Adapted from Bell et al. 2004.

 

Long Description: The exhibit shows the instances when the initial standards work in each of the five medication management steps/activities in the process model of medication management, which includes:  prescribe, transmit, dispense, administer, and monitor.

 

Step 3, “dispense,” involves a pharmacist directly except when medications are dispensed in the clinician’s office.  Pharmacists may access the same types of information and requires the same standards used by physicians or their surrogates in the prescribe step.  Problematic prescriptions may require a call to clinical staff, as a result, prescriptions may be changed or cancelled rather than dispensed.  Step 4, “administer,” involves the provision of educational information to the patient which explains how to take the medication.  In this step, problematic prescriptions may again require a call to clinical staff, thus the prescription could be changed or cancelled.  Finally, step 5, “monitor,” includes the assessment by clinical staff for changes to prescriptions depending on the patient’s reaction to the prescription.  This step can also involve the NCPDP script transactions, as well as medication fill status messaging.

Interoperability is meant to ensure the rapid flow of secure, private and complete digitized information across all of these tasks of care.  It is not enough for each of the standards to work on their own; they must work together smoothly, or “interoperate.” There are different classes of standards, some that define all the data elements needed, some for unique identification of participants in the message exchange, as well as standards that provide guidelines for clinical protocols and care.  To ensure interoperability with other e-health applications a common set of data elements including the standard types listed in Exhibit 8 below must be adopted. [xlix]

 

Exhibit 8.     Standard for Interoperable Health Care

From the Data Standards Work Group efforts on Building a Standards for an Interoperable Health Care System: Connecting for Health, Markle Foundation

 

Long Description: The exhibit illustrates the various components required for e-prescribing to work effectively, including:  e-health applications, a common set of data elements that include the standards themselves, the system’s architecture, security, and data indexing and validation features or meta-elements.

At the center of the model are data messaging and application-related standards which are a primary focus of the pilot testing.  Data messaging standards enable consistent communication between applications within an institution and across user organizations. The application-related standards support consistent and efficient implementation of specifications, enable integration of applications, and provide software components that can be implemented in different applications.

The standards also include meta-elements such as vocabularies and code sets.  The pilot sites were charged with determining whether these standards were unequivocal and could communicate needed information, demonstrating interoperability with those standards named as foundations standards.

Validation is a key component in understanding how standards work accurately ensuring the consistency of implementation of standards across the various software applications and user implementations.  Validation is used in reference to the activity of checking the adherence to standards, a primary objective of the pilot testing.

Interoperability is dependent upon all requisite standards being identified, adopted and implemented in a consistent manner.  Manuals or implementation guides are essential to allow for easy and timely implementations.  Many of the foundation standards have been adopted and implemented widely throughout the industry although not necessarily in a consistent manner, which leaves the impression that there is a lack of data standards.  The pilot sites have put this theory into practice over the past year.  Their findings and efforts should enable the establishment of best practices for certification and implementation that will move the e-prescribing industry forward. 


This section will summarize the key objectives, organizational characteristics, and various approaches to standards testing and evaluating outcomes taken by each of the five pilot sites.

Overview of the CMS/AHRQ Pilot Site Portfolio

One of the strengths of the pilot testing was the diversity and uniqueness of the five pilot sites.  Pilot sites represented the spectrum of communities involved with e-prescribing, including most practice settings, and focusing on pharmacists, physicians, nurses, technology vendors, and medical assistants (see Exhibit 9).

Exhibit 9.     Settings and Stakeholders Included in Pilot Sites