With all of the research and investment dedicated to artificial intelligence and other automation technologies, there is a paucity of evaluation methods for how these technologies integrate into effective joint human-machine teams. Current evaluation methods, which largely were designed to measure performance of discrete representative tasks, provide little information about how the system will perform when operating outside the bounds of the evaluation. We are exploring a method of generating Extensibility Plots, which predicts the ability of the human-machine system to respond to classes of challenges at intensities both within and outside of what was tested. In this paper we test and explore the method, using performance data collected from a healthcare setting in which a machine and nurse jointly detect signs of patient decompensation. We explore the validity and usefulness of these curves to predict the graceful extensibility of the system.

Even as vaccination for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) expands in the United States, cases will linger among unvaccinated individuals for at least the next year, allowing the spread of the coronavirus to continue in communities across the country. Detecting these infections, particularly asymptomatic ones, is critical to stemming further transmission of the virus in the months ahead. This will require active surveillance efforts in which these undetected cases are proactively sought out rather than waiting for individuals to present to testing sites for diagnosis. However, finding these pockets of asymptomatic cases (i.e., hotspots) is akin to searching for needles in a haystack as choosing where and when to test within communities is hampered by a lack of epidemiological information to guide decision makers’ allocation of these resources. Making sequential decisions with partial information is a classic problem in decision science, the explore v. exploit dilemma. Using methods—bandit algorithms—similar to those used to search for other kinds of lost or hidden objects, from downed aircraft or underground oil deposits, we can address the explore v. exploit tradeoff facing active surveillance efforts and optimize the deployment of mobile testing resources to maximize the yield of new SARS-CoV-2 diagnoses. These bandit algorithms can be implemented easily as a guide to active case finding for SARS-CoV-2. A simple Thompson sampling algorithm and an extension of it to integrate spatial correlation in the data are now embedded in a fully functional prototype of a web app to allow policymakers to use either of these algorithms to target SARS-CoV-2 testing. In this instance, potential testing locations were identified by using mobility data from UberMedia to target high-frequency venues in Columbus, Ohio, as part of a planned feasibility study of the algorithms in the field. However, it is easily adaptable to other jurisdictions, requiring only a set of candidate test locations with point-to-point distances between all locations, whether or not mobility data are integrated into decision making in choosing places to test.

Despite the promise of a proactive approach to safety, a lack of resources and tangible measures have limited its implementation in organizations. We are exploring Joint Activity Monitoring (JAM) as one key component of a proactive safety program within the domain of infection prevention. However, despite a conceptual alignment to the requirements of a proactive monitoring capability, our experiences instrumenting daily work tools with the capabilities to support continuous, unobtrusive, real-time monitoring have revealed additional organizational and technological requirements. In this paper, we describe our strategies and challenges in developing this capability and discuss implications for supporting successful proactive safety implementations.

This panel discussion will examine the societal awareness of cognitive engineering today. Cognitive engineering celebrated its 30th anniversary in 2018 at the HFES annual meeting. Still, some would say that cognitive engineering is not as well-known as it should be, and that it is applied in an ad hoc manner in the many high-stakes, high-risk technology modernization efforts where it would be useful. As technology advances proliferate for sharp end of the spear decision makers, we are at risk of catastrophic results if CE remains in the shadows; these results are arguably emerging on a daily basis. Each panelist will describe, from their vantage point, CE’s state of the art today, thoughts on barriers to acceptance and application, and how they envision we act towards a future in 2028 in which cognitive engineers engage systematically in complex systems’ development.

We introduce the concept of machine fitness assessment, which is the process of correctly determining the degree of fit between a machine’s inferences on a specific world and the world itself. We describe its importance in complex, high-stakes worlds, including healthcare, and how it will be critically important to realize the potential of consumer health technologies that promise institutional-quality health diagnosis and planning in decidedly non-institutional settings (e.g., our homes, offices, or anywhere else).