Development and Evaluation of Sociotechnical Metrics To Inform Health IT Adaptation

The United States has undergone a rapid adoption of health information technology (IT). Adopting new technology requires the redesign of individual and collective workflows and results in changes in both organizational structure and processes. Historically, adoption of health IT, such as electronic health records (EHRs), has been evaluated using adoption rates, acceptance, and clinical quality measures. Although these measures are helpful to assess the maturity of implementation, they do not inform organizational or human factors that could delay or cause the intent of health IT to fail. Because implementation is a process and not an outcome, adoption should be assessed through understanding of the progress of adaptation from a sociotechnical perspective. Sociotechnical theory explains the interdependent relationship between humans, organizations, and technology. The organizational and human (socio) elements and IT elements (technical) are interdependent parts of a system that must be evaluated together and should not be studied in isolation.

This research sought to develop a fundamental and multi-dimensional survey instrument that can inform the progress of health IT adaptation informed by the socio-technical theory to address health IT in complex adaptive healthcare systems.

The specific aim of this research was as follows:

• Identify measures and develop instruments to evaluate the progress of health IT adaptation 

The investigators used a mixed-methods approach to incorporate quantitative and qualitative research in developing and validating a health IT adaptation survey instrument. They followed the National Institutes of Health Patient-Reported Outcomes Measurement Information System (PROMIS®) model, a nine-step standard considered best practice for instrument development and validation. The survey development was guided and informed by the socio-technical theory as well as semi-structured interviews with different stakeholders, including clinicians, leaders, trainers, and informaticians.

Major themes of the analyses include superusers and peers for onsite support, resistance and workarounds, involvement of health IT development and upgrades, user competency, feedback on performance, and leadership. The findings informed the development of the health IT adaptation survey instrument, which incorporates six factors: usability, communication, information dissemination, training, help desk, and leadership. The investigators suggest that organizations can use the survey instrument to identify challenging areas and to investigate further areas that weaken or delay clinicians’ adaptation.