Force Health Surveillance in NATO Does Not Meet the Needs of Its Users: A Structured Evaluation of EpiNATO-2

Abstract

Introduction Disease and non-battle injuries (DNBIs) cause substantial losses among military personnel. NATO has monitored DNBIs among its personnel since 1996 using multiple versions of a tool now called EpiNATO-2, but the surveillance system has never been systematically evaluated. Following a request from NATO to the CDC, the objective of this study was to assess surveillance system attributes of EpiNATO-2 using CDC’s updated guidelines for evaluating public health surveillance systems. Materials and Methods Between June and October 2022, a literature review and key informant interviews were conducted to assess the following attributes: usefulness, simplicity, flexibility, data quality, acceptability, sensitivity, positive predictive value, representativeness, timeliness, stability, informatics system quality, informatics service quality, and informatics interoperability. Key informant interviews were conducted in Kosovo, Germany, and remotely with EpiNATO-2 users spanning three levels: clinical and data entry personnel (tactical level); regional medical and public health officers (operational level); and senior commanders and other governmental entities (strategic level). Results Fourteen EpiNATO-2 users participated in interviews, representing 3 of the 5 major NATO missions, 3 partner entities, and 7 nationalities. All users (100%) reported that the system did not meet their needs, with most users noting the following challenges: lack of clearly defined system objectives; poor data quality due to ambiguous case definitions and frequently unsubmitted reports (37% missing during January to June 2022); long delay between the occurrence of health events and the availability of corresponding data (≥2 weeks); and an antiquated and inflexible data management system. Overall, performance was deemed unsatisfactory on 11 of the 13 attributes. Conclusions This multinational sample of EpiNATO-2 users at all military levels reported that the system is currently not useful with respect to its stated objectives. Opportunities exist to improve the performance and usefulness of EpiNATO-2: improve case definitions, modernize data infrastructure, and regularly evaluate the surveillance system.