Fingers of a Hand Oscillate Together

Williamson, John

doi:10.1145/2858036.2858235

Cited by 5 publications

(1 citation statement)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[ Hinckley et al 2016] demonstrate 'pre-touch', above-surface interaction in a mobile phone sensing configuration similar to that used in this paper. [Williamson 2016] analyses above-device finger tremor characteristics using capacitive sensing. [Xiao et al 2015] used classical machine learning techniques to infer 3D pose.…”

Section: Previous 3d Capacitive Workmentioning

confidence: 99%

Forward and Inverse models in HCI:Physical simulation and deep learning for inferring 3D finger pose

Murray-Smith,

Williamson,

Ramsay

et al. 2021

Preprint

View full text Add to dashboard Cite

We outline the role of forward and inverse modelling approaches in the design of human-computer interaction systems. Causal, forward models tend to be easier to specify and simulate, but HCI requires solutions of the inverse problem. We infer finger 3D position (x, y, z) and pose (pitch and yaw) on a mobile device using capacitive sensors which can sense the finger up to 5cm above the screen. We use machine learning to develop data-driven models to infer position, pose and sensor readings, based on training data from: 1. data generated by robots, 2. data from electrostatic simulators 3. human-generated data. Machine learned emulation is used to accelerate the electrostatic simulation performance by a factor of millions. We combine a Conditional Variational Autoencoder with domain expertise/models experimentally collected data. We compare forward and inverse model approaches to direct inference of finger pose. The combination gives the most accurate reported results on inferring 3D position and pose with a capacitive sensor on a mobile device.CCS Concepts: •Human-centered computing → HCI theory, concepts and models;

show abstract

Section: Previous 3d Capacitive Workmentioning

confidence: 99%