Development and Psychometric Qualities of the SEIPS Survey to Evaluate CPOE/EHR Implementation in ICUs

Abstract

Health Information Technology (IT) implementation can fail or meet high levels of user resistance for a variety of reasons, including lack of attention to users' needs and the significant workflow changes induced and required by the technology. End-user satisfaction is a critical factor in health IT implementation. In this paper we describe the process of developing and testing a questionnaire to evaluate health IT implementation, in particular Computerized Provider Order Entry (CPOE) and Electronic Health Record (EHR) technologies. Results showed evidence for the validity and reliability of the questionnaire. The Systems Engineering Initiative for Patient Safety (SEIPS) questionnaire is relatively easy to administer and allows researchers to evaluate different aspects of health IT implementation. Results of this research can be used for benchmarking results of future studies evaluating health IT implementation.

Human factors systems approach to healthcare quality and patient safety

“…The lack of attention to HFE in the design, implementation, and use of health IT contributes to limited success of health IT, the failures to implement and sustain health IT, and safety problems linked to health IT (Karsh et al, 2010). Our research has examined the impact of health IT on work system and patient safety in a variety of domains: BCMA technology in hospital settings (Carayon et al, 2007b; Koppel et al, 2008), smart infusion pump technology in a hospital (Carayon et al, 2010; Schroeder et al, 2006; Wetterneck et al, 2006), and CPOE/EHR in ICUs (Carayon et al, 2011c; Hoonakker et al, 2010; Hoonakker et al, 2011; Wetterneck et al, 2011). We have also used the SEIPS model to classify self-reported facilitators and barriers to physicians’ use of implemented CPOE/EHR (Holden, 2011c) and to improve the design of a clinical decision support for primary care physicians (Hoonakker et al, 2012).…”

“…The SEIPS model is the conceptual framework for a range of HFE tools and methods used to evaluate healthcare work systems and processes and their impact on healthcare quality and patient safety, such as survey questionnaire (Carayon et al, 2005a; Hoonakker et al, 2010; Hoonakker et al, 2011), observation methodology (Carayon et al, 2005d; Carayon et al, 2007b), interviews (Carayon et al, 2006a; Gurses and Carayon, 2009), and proactive risk assessment of healthcare processes (Carayon et al, 2011e; Hundt et al, 2013; Wetterneck et al, 2009; Wetterneck et al, 2006). …”

Sociotechnical systems approach to healthcare quality and patient safety

“…The SEIPS model is the conceptual framework for a range of HFE tools and methods used to evaluate healthcare work systems and processes and their impact on healthcare quality and patient safety, such as questionnaire survey [4, 23, 24], observation methodology [13, 14], interviews [3, 20], and proactive risk assessment of healthcare processes [8, 38, 40]. …”

“…A reason for the limited success of health IT, the failures to implement and sustain health IT, and safety problems linked to health IT is the lack of attention to HFE in the design, implementation and use of health IT. Our research has examined the impact of health IT on work system and patient safety in a variety of domains: bar coding medication administration technology in hospital settings [14, 27], smart infusion pump technology in a hospital [10, 32, 39], and CPOE/EHR in ICUs [6, 23, 24, 41]. …”

Mind the Gap

“…Two articles described survey-based methods to identify problem areas during implementation and go-live. One survey tool, based on the Systems Engineering Initiative for Patient Safety model (SEIPS), focused on evaluating EHR implementation, and can be used to assess usefulness and satisfaction [36]. The other survey described focused on user experience to identify areas for improvement including the concept domains: training and competency, usability, infrastructure, usefulness, and end-user support [37].…”