Real-Time Speech Workload Estimation for Intelligent Human- Machine Systems

Fortune, Julian; Heard, Jamison; Adams, Julie A.

doi:10.1177/1071181320641076

Cited by 3 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several speech-related metrics (e.g., speech rate, pitch, and voice intensity) that do not rely on natural language processing have proven effective for estimating speech workload ( Heard et al, 2018a ; Heard et al, 2019a ; Fortune et al, 2020 ). Speech rate captures verbal communications’ articulation rate by measuring the number of syllables uttered per unit time ( Fortune et al, 2020 ). Voice intensity is the speech signal’s root-mean-square value, while Pitch is the signal’s dominant frequency over a time period ( Heard et al, 2019a ).…”

Section: Task Recognition Metricsmentioning

confidence: 99%

Multi-dimensional task recognition for human-robot teaming: literature review

Baskaran,

Adams

2023

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

Human-robot teams collaborating to achieve tasks under various conditions, especially in unstructured, dynamic environments will require robots to adapt autonomously to a human teammate’s state. An important element of such adaptation is the robot’s ability to infer the human teammate’s tasks. Environmentally embedded sensors (e.g., motion capture and cameras) are infeasible in such environments for task recognition, but wearable sensors are a viable task recognition alternative. Human-robot teams will perform a wide variety of composite and atomic tasks, involving multiple activity components (i.e., gross motor, fine-grained motor, tactile, visual, cognitive, speech and auditory) that may occur concurrently. A robot’s ability to recognize the human’s composite, concurrent tasks is a key requirement for realizing successful teaming. Over a hundred task recognition algorithms across multiple activity components are evaluated based on six criteria: sensitivity, suitability, generalizability, composite factor, concurrency and anomaly awareness. The majority of the reviewed task recognition algorithms are not viable for human-robot teams in unstructured, dynamic environments, as they only detect tasks from a subset of activity components, incorporate non-wearable sensors, and rarely detect composite, concurrent tasks across multiple activity components.

show abstract

Section: Task Recognition Metricsmentioning

confidence: 99%

Multi-dimensional task recognition for human-robot teaming: literature review

Baskaran,

Adams

2023

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

show abstract

“…Heard and Adams' multidimensional workload algorithm estimates a human's workload components and the composite overall workload state (Fortune et al, 2020;Heard et al, 2019a;Heard et al, 2019b;Heard and Adams, 2019;Heard, 2019). This algorithm was developed specifically to support unstructured dynamic domains (e.g., disaster response, military) using primarily wearable, nonvision based sensors that can objectively measure the human's current performance (e.g., overall workload (Heard and Adams, 2019;Heard et al, 2019b)).…”

Section: Multidimensional Workload Algorithm Overviewmentioning

confidence: 99%

“…The multidimensional workload component states (i.e., auditory, cognitive, physical, speech, and visual (McCraken and Aldrich, 1984)) are estimated and are used to estimate and classify overall workload (i.e., underload, normal load, and overload). The algorithm incorporates objective physiologically-based metrics, available via wearable sensors, and a nonphysiological environmental metric that correlate to overall workload and the multidimensional components (Fortune et al, 2020;Harriott et al, 2013;Harriott et al, 2015;Heard et al, 2018;Heard and Adams, 2019;Heard et al, 2019b;Heard et al, 2019a).…”

Section: Multidimensional Workload Algorithm Overviewmentioning

confidence: 99%

“…The multidimensional workload algorithm estimates overall workload and its components by extracting time based features (i.e., mean, variance, average gradient, and slope) from thirty second epochs for each objective workload metric (e.g., heart-rate variability, posture magnitude, noise level). The time based features serve as inputs to a corresponding neural network that estimates each workload component (Fortune et al, 2020;Heard and Adams, 2019;Heard, 2019). The means and standard deviations capture the metrics' response to workload variations, but do not capture a metric's directional shift, (e.g., the metric is increasing over the time window).…”

Section: Multidimensional Workload Algorithm Overviewmentioning

confidence: 99%

“…The component estimates are uniformly aggregated to estimate overall workload, which was mapped to a state (i.e., underload, normal load, or overload) using thresholds. Heard et al conducted extensive validation of the multidimensional algorithm across supervisory and peer-based relationships, tasks, workload conditions and populations (Fortune et al, 2020;Heard et al, 2019a;Heard et al, 2019b;Heard and Adams, 2019;Heard, 2019). These validations used IMPRINT Pro models, developed prior to conducting human subjects evaluations, as the comparison to the multidimensional workload algorithm's results.…”

Section: Multidimensional Workload Algorithm Overviewmentioning

confidence: 99%

See 2 more Smart Citations

Can A Single Human Supervise A Swarm of 100 Heterogeneous Robots?

Adams,

Hamell,

Walker

2023

View full text Add to dashboard Cite

An open research question has been whether a single human can supervise a true heterogeneous swarm of robots completing tasks in real-world environments. A general concern is whether or not the human’s workload will be taxed to the breaking point. The Defense Advanced Research Projects Agency’s OFFensive Swarm-Enabled Tactics (OFFSET) program’s field exercises that occurred at U.S. Army urban training sites provided the opportunity to understand the impact of achieving such swarm deployments. The Command and Control of Aggregate Swarm Tactics integrator team’s swarm commander uses the heterogeneous robot swarm to conduct relevant missions. During the final OFFSET program field exercise, the team collected objective and subjective metrics related to the swarm commander’s human performance. A multidimensional workload algorithm that estimates the overall workload based on five components of workload was used to analyze the results. While the swarm commander’s workload estimates did cross the overload threshold frequently, the swarm commander was able to successfully complete the missions, often under challenging operational conditions. The presented results demonstrate that a single human can deploy a swarm of 100 heterogeneous robots to conduct real-world missions.

show abstract