Methods

Key Questions | Technical Expert Panel | Literature Search | Article Review

Key Questions

1. What does the evidence show with respect to the costs and benefits of inter-operable electronic HIT data exchange for providers and payers/purchasers?
2. What is a framework that could be used in this study to describe levels/bundles of EHR functionality and to estimate the costs and benefits by such levels/bundles of functionality by payer/purchaser and percentage of provider penetration?
3. What knowledge or evidence deficits exist regarding needed information to support estimates of cost, benefit and net value with regard to HITsystems? Discuss gaps in research, including specific areas that should be addressed, and suggest possible public and private organizational types to perform the research and/or analysis.
4. What critical cost/benefit information is required by decision makers (at various levels) in order to give a clear understanding of HIT Systems value proposition particular to them?
5. What analytic methods (e.g., sources of data, algorithms, etc.) could be used to produce evidence of the costs and benefits within and across health care provider settings, payers/purchasers, and cumulatively across the health care delivery continuum and payers, of deploying electronic health information technology functions examined in this study?
6. What are the barriers that health care providers and health care systems encounter that limit implementation of electronic health information systems?

Technical Expert Panel

This project was advised by a technical expert panel. Service as an expert panel member should not in any way be construed as agreeing with or endorsing the content of the report or database.

Literature Search Methods

The following piece describes how we found the articles to include in this database.
We were given a list of titles from a prior project on information technology in health care. That project conducted a search of PubMed in November 2003. They searched for systematic reviews published in English from 1995 to 2003. PubMed, which is maintained by the U.S. National Library of Medicine, is widely recognized as the premier source for bibliographic coverage of biomedical literature. It encompasses information from Index Medicus, the Index to Dental Literature, and the Cumulative Index to Nursing and Allied Health Literature (allied health includes occupational therapy, speech therapy, and rehabilitation), as well as other sources of coverage in the areas of health care organization, biological and physical sciences, humanities, and information science as they relate to medicine and health care.

Our own search for studies of health information technology began with an electronic search of PubMed on January 6, 2004 for additional articles published since 1995. We ordered all articles on HIT topics, regardless of study design or language. We also searched the Cochrane Controlled Clinical Trials Register Database and the Cochrane Database of Reviews of Effectiveness (DARE). The Cochrane Collaboration is an international organization that helps people make well-informed decisions about health care by preparing, maintaining, and promoting the accessibility of systematic reviews on the effects of heath care interventions. In December, 2004 we also conducted a specific search of the journal Health Affairs , developing a list of all articles with "information technology" or "information systems" as keywords. Health Affairs has been a leader in publishing articles on this topic in recent years.

We also selected references from the following published reports:
Advanced Technologies to Lower Health Care Costs and Improve Quality - This report was published in the fall of 2003 by the Massachusetts Technology Collaborative in partnership with the New England Healthcare Institute. Research was conducted by the First Consulting Group and was sponsored by several Massachusetts companies involved in healthcare and health insurance. The report focuses on seven advanced technologies (computerized physician order entry, electronic prescribing, etc) that have demonstrated both financial benefits and improved qualityof care. The report also discusses barriers to implementation.

The Value of Computerized Provider Order Entry (CPOE) in Ambulatory Settings - The Center for Information Technology, also located in the Boston area, published this report in 2003. They searched internationally for both academic and commercial sources of literature. They also contacted 35 vendors regarding their currently available health information technology packages. They found that CPOE can significantly improve quality while lowering costs.

Published Meta-analysis on Computer-based Clinical Reminder Systems - In 1996, Shea, DuMouchel, and Bahamonde published a meta-analysis of 16 trials in the Journal of the American Medical Informatics Association (JAMIA). They found that computer reminders improved utilization of vaccinations, breast cancer screenings, and colorectal cancer screenings, but not pap smears or other preventive care.

Article Review

This piece describes the review process and the article "study design" categories.
We reviewed the articles retrieved from the various sources to determine whether to include them in the interactive database. A one-page screening review form that contained a series of categorization questions was created to track the articles. Two reviewers, each trained in the critical analysis of scientific literature, independently reviewed each study, and resolved disagreements by consensus. The principal investigator resolved any disagreements that remained unresolved after discussions among the reviewers.

The database includes evidence from articles with many different study designs. Our initial search was unrestricted by study design. We identified review articles and further classified these as systematic or non-systematic reviews. We classified other articles as descriptive if they primarily described the workings or implementation of an HIT system and further classified these as qualitative or quantitative, depending on the presentation of numerical values for things like numbers of tests ordered, costs of implementation, etc. A second category of studies we classified as "hypothesis testing" by which we meant that a study question was assessed by comparing data between groups or across time periods and using statistical test to assess differences. Hypothesis testing studies were further classified as containing an intervention with a concurrent comparison group which included randomized and non-randomized controlled trials and controlled before-after studies; and studies with an intervention but without a concurrent comparison group; which included pre-post studies, time series studies with more than two measurement points, and studies that used a historical control group. Additional classifications of hypothesis studies testing included studies without an intervention which were cross-sectional in nature, and other hypothesis testing studies. Lastly, we classified studies as a "predictive analysis" if they used modeling techniques to predict what might happen with an HIT implementation rather than what has happened with an HIT implementation. Predictive analyses include cost effectiveness and cost benefit analyses, and typically use data from multiple studies and are dependent upon several assumptions, some of which are not always explicitly stated. Articles that were classified as systematic reviews or meta-analyses, hypothesis testing or predictive analyses had structured abstracts created and included in this interactive database.