Non-Verbal Auditory Input for Controlling Binary, Discrete, and Continuous Input in Automotive User Interfaces

Funk, Martin; Tobisch, Vanessa; Emfield, Adam

doi:10.1145/3313831.3376816

Cited by 13 publications

(5 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Various studies have investigated the potential of NVVEs as a means of providing accessible input mechanisms [11,7,12]. Non-verbal voice expressions can be detected and when detected undergo binary classification.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

CNVVE: Dataset and Benchmark for Classifying Non-verbal Voice

Hedeshy¹,

Menges²,

Staab³

2023

Interspeech 2023

View full text Add to dashboard Cite

Non-verbal voice expressions (NVVEs) have been adopted as a means of human-computer interaction in research studies. However, exploring non-verbal voice-based interactions has been constrained by the limited availability of suitable training data and computational methods for classifying such expressions, leading to a focus on simple binary inputs. We address this issue with a new dataset containing 950 audio samples comprising 6 classes of voice expressions. The data were collected from 42 speakers who donated voice recordings. The classifier was trained on the data using features derived from mel-spectrograms. Furthermore, we studied the effectiveness of data augmentation and improved over the baseline model accuracy significantly with a test accuracy of 96.6% in a 5-fold cross-validation. We have made CNVVE publicly accessible in the hope that it will serve as a benchmark for future research.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In this way, they can be used as on/off signals, comparable to a button [13]. Continuous non-verbal voice expressions, such as humming, can be used for a slider [11], or in combination with eye-tracking for hands-free text entry [5]. Additionally, NVVE can serve as an alternative input modality for wheelchair control [4].…”

Section: Related Workmentioning

confidence: 99%

CNVVE: Dataset and Benchmark for Classifying Non-verbal Voice

Hedeshy¹,

Menges²,

Staab³

2023

Interspeech 2023

View full text Add to dashboard Cite

show abstract

“…The visual category entails the eye as a sensor enabling passengers to perceive any light-based output modalities and as an actuator that produces explicit input modalities, e.g., gaze, pupil dilation, or blink rate [340]. Auditory output modalities are sensed by the ears, while auditory actuators are body parts that can explicitly generate sounds, e.g., mouth for voice or hands/fingers for clapping [111]. According to Benyon [33] and the ISO standard 9241-910 [272], we divide haptic into kinesthetic, cutaneous, and vestibular.…”

Section: Human Sensors and Actuatorsmentioning

confidence: 99%

“…Furthermore, non-speech sounds generated by the vocal tract or body parts were used. For example, Funk et al [111] enabled nonverbal auditory input (humming, snapping, and clapping) for interaction with an assistant while driving.…”

Section: Input Modalitiesmentioning

confidence: 99%

“…For example, cameras for visual input, such as gaze [116], blinking [240], or pupil dilation [240], or kinesthetic input, such as gesture [81], facial expression [153], or head movement [314]. Similarly, microphones were placed in the front to sense auditory input (e.g., speech [245] or humming [111]). Other input modalities often require direct contact with an anchored interior location.…”

Section: Interaction Locationsmentioning

confidence: 99%

See 1 more Smart Citation

A Design Space for Human Sensor and Actuator Focused In-Vehicle Interaction Based on a Systematic Literature Review

Jansen

Colley

Rukzio

2022

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

View full text Add to dashboard Cite

Automotive user interfaces constantly change due to increasing automation, novel features, additional applications, and user demands. While in-vehicle interaction can utilize numerous promising modalities, no existing overview includes an extensive set of human sensors and actuators and interaction locations throughout the vehicle interior. We conducted a systematic literature review of 327 publications leading to a design space for in-vehicle interaction that outlines existing and lack of work regarding input and output modalities, locations, and multimodal interaction. To investigate user acceptance of possible modalities and locations inferred from existing work and gaps unveiled in our design space, we conducted an online study (N=48). The study revealed users' general acceptance of novel modalities (e.g., brain or thermal activity) and interaction with locations other than the front (e.g., seat or table). Our work helps practitioners evaluate key design decisions, exploit trends, and explore new areas in the domain of in-vehicle interaction.

show abstract