No abstract

The association of tabletop interaction with gesture typing presents interaction potential for situationally or physically impaired users. In this work, we use depth cameras to create touch surfaces on regular tabletops. We describe our prototype system and report on a supervised learning approach to fingertips touch classification. We follow with a gesture typing study that compares our system with a control tablet scenario and explore the influence of input size and aspect ratio of the virtual surface on the text input performance. We show that novice users perform with the same error rate at half the input rate with our system as compared to the control condition, that an input size between A5 and A4 present the best tradeoff between performance and user preference and that users' indirect tracking ability seems to be the overall performance limiting factor.

One of the challenges of technology-assisted motor learning is how to adapt practice to facilitate learning. Random practice has been shown to promote long-term learning. However, it does not adapt to the learner's specific learning requirements. Previous attempts to adapt learning consider the skill level of learners from past training sessions. This study investigates the effects of personalizing practice in real time, through a Curriculum Learning approach, where a curriculum of tasks is built by considering consecutive performance differences for each task. 12 participants were allocated to each of three training conditions in an experiment which required performing a steering task to drive a cursor in an arc channel. The Curriculum Learning approach was compared to two other conditions: random practice and another adaptive practice, which does not consider the learning evolution. The Curriculum Learning practice outperformed the random practice in effectively decreasing movement variability at post-test and outperformed both the random practice and the adaptive practice on transfer tests. The adaptation of Curriculum Learning practice also made learners' skills more uniform. Based on these findings, we anticipate that future research will explore the use of Curriculum Learning in interactive training tools to support motor skill learning, such as rehabilitation.

Analysing movement learning can rely on human evaluation, e.g. annotating video recordings, or on computing means in applying metrics on behavioural data. However, it remains challenging to relate human perception of movement similarity to computational measures that aim at modelling such similarity. In this paper, we propose a metric learning method bridging the gap between human ratings of movement similarity in a motor learning task and computational metric evaluation on the same task. It applies metric learning on a Dynamic Time Warping algorithm to derive an optimal set of movement features that best explain human ratings. We evaluated this method on an existing movement dataset, which comprises videos of participants practising a complex gesture sequence toward a target template, as well as the collected data that describes the movements. We show that it is possible to establish a linear relationship between human ratings and our learned computational metric. This learned metric can be used to describe the most salient temporal moments implicitly used by annotators, as well as movement parameters that correlate with motor improvements in the dataset. We conclude with possibilities to generalise this method for designing computational tools dedicated to movement annotation and evaluation of skill learning.