Next-Point Prediction for Direct Touch Using Finite-Time Derivative Estimation

Abstract

End-to-end latency in interactive systems is detrimental to performance and usability, and comes from a combination of hardware and software delays. While these delays are steadily addressed by hardware and software improvements, it is at a decelerating pace. In parallel, short-term input prediction has shown promising results in recent years, in both research and industry, as an addition to these efforts. We describe a new prediction algorithm for direct touch devices based on (i) a state-of-the-art finite-time derivative estimator, (ii) a smoothing mechanism based on input speed, and (iii) a post-filtering of the prediction in two steps. Using both a preexisting dataset of touch input as benchmark, and subjective data from a new user study, we show that this new predictor outperforms the predictors currently available in the literature and industry, based on metrics that model user-defined negative side-effects caused by input prediction. In particular, we show that our predictor can predict up to 2 or 3 times further than existing techniques with minimal negative side-effects.

Modeling and Reducing Spatial Jitter caused by Asynchronous Input and Output Rates

“…Android and Chromium, for instance, use first-order Taylor series. Other techniques like Kalman filters, or machine learning approaches trained for fixed latencies [23], are designed to predict fixed time horizons. Nancel et al provide a detailed review of each technique [24].…”

“…Spatial jitter in next-point prediction [24] is a consequence of (1) input sensing noise and quantization that can make speed and direction estimation inaccurate, and (2) errors in extrapolating an ongoing trajectory over several frames [23]. However, jitter can occur even with zero input noise and a perfect prediction model -or with no prediction at all.…”

“…when scrolling text while skimming to locate a particular word, but possibly with smaller viewport jumps. One also needs to consider that noise occurring at high velocity can be less noticeable than at low velocity [10,23]. This theoretical trade-off between amplitude and duration is discussed later in the context of a real-case example.…”

“…d pre values therefore increase the chances of extrapolation, which in turn can introduce "prediction" side-effects [24,23] (including more jitter). Interpolation offers smooth transitions between known input values, but increases the delay between input and output signals.…”

“…Such techniques can be used for instance to predict the next few points of a movement in or der to compensate latencies [19,23], allowing for different degrees of accuracy and negative side-effects depending on the predicted duration [24], or "horizon". In particular, sec ond order polynomial curve fitting (dubbed CU RV E in [24]) and Double-Exponential Smoothing Predictor (DESP) from LaViola [19] were found to generate fewer side-effects overall compared to other predictors [23]. Other prediction techniques such as Kalman filters or the TurboTouch Predictor [23] can also offer reliable predictions, but are designed to predict at fixed horizons.…”