Designing for low-latency direct-touch input

Ng, Albert; Lepinski, Julian; Wigdor, Daniel; Sanders, S.; Dietz, Paul

doi:10.1145/2380116.2380174

Cited by 162 publications

(127 citation statements)

References 22 publications

Supporting

Mentioning

124

Contrasting

Order By: Relevance

“…This end-to-end latency can be split into hardware latency (on input side: sensing, transmission and on output side : display) and software latency (system, toolkits, frameworks and applications). End-to-end latency is still affecting most interactive systems [9,27,30], from desktop computers (where latency is between 50 and 90 ms [9]) to touch-based devices (typically, latency on touch-screens is between 60 and 200 ms [10,30]). And while modern input controllers (e.g.…”

Section: Introductionmentioning

confidence: 99%

Using High Frequency Accelerometer and Mouse to Compensate for End-to-end Latency in Indirect Interaction

Antoine

Malacria

Casiez

2018

Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

End-to-end latency corresponds to the temporal difference between a user input and the corresponding output from a system. It has been shown to degrade user performance in both direct and indirect interaction. If it can be reduced to some extend, latency can also be compensated through software compensation by trying to predict the future position of the cursor based on previous positions, velocities and accelerations. In this paper, we propose a hybrid hardware and software prediction technique specifically designed for partially compensating endto-end latency in indirect pointing. We combine a computer mouse with a high frequency accelerometer to predict the future location of the pointer using Euler based equations. Our prediction method results in more accurate prediction than previously introduced prediction algorithms for direct touch. A controlled experiment also revealed that it can improve target acquisition time in pointing tasks.

show abstract

Section: Introductionmentioning

confidence: 99%

Using High Frequency Accelerometer and Mouse to Compensate for End-to-end Latency in Indirect Interaction

Antoine

Malacria

Casiez

2018

Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…In OST systems, the real world is viewed directly, which makes the latency detectable at considerably lower levels. A study by Ng et al [29] found the JND of latency to be as low as 2.38 ms for OST systems.…”

Section: Latency Detectionmentioning

confidence: 97%

Driving in Virtual Reality: Investigations in Effects of Latency and Level of Virtuality

Blissing¹

2016

View full text Add to dashboard Cite

When developing new active safety systems or improving existing systems, conducting performance evaluations is necessary. By performing these evaluations during early development stages, potential problems can be identified and mitigated before the system moves into the production phase.Testing active safety systems can be difficult since the characteristic scenarios may have complex interactions. Using real vehicles for performing these types of scenarios is difficult, expensive, and potentially dangerous. Alternative methods, such as using inflatable targets, scale models, computer simulations or driving simulators, also suffer from drawbacks. Consequently, using virtual reality as an alternative to the traditional methods has been proposed. In this case, a real vehicle is driven while wearing a head-mounted display that presents the scenario to the driver.This research aims to investigate the potential of such technology. Specifically, this work investigates how the chosen technology affects the driver. This investigation has been conducted through a literature review. A test platform was constructed, and two user studies using normal drivers were performed. The first study focused on the effects of visual time delays on driver behavior. This study revealed that lateral behavior changes with added time delays, whereas longitudinal behavior appears unaffected. The second study investigated how driver behavior is affected by different modes of virtuality. This study demonstrated that drivers perceived mixed reality as more difficult than virtual reality.The main contribution of this work is the detailed understanding of how time delays and different modes of virtuality affect drivers. This is important knowledge for selecting which scenarios are suitable for evaluation using virtual reality. Keywords PapersThe following four appended papers are arranged in chronological order of publication and will be referred to by their Roman numerals. All papers are printed in their original published state with the exception of minor errata and changes in text and figure layout in order to maintain consistency throughout the thesis. In papers [I], [II], [III] and [IV], the first author is the main author, responsible for the work presented, with additional support from the co-authors. A short summary of each paper can be found in chapter 4.[I] Björn Blissing, Fredrik Bruzelius, and Johan Ölvander." [VIII] Lars Eriksson, Lisa Palmqvist, Jonas Andersson Hultgren, Björn Blissing, and Steven Nordin. "Performance and presence with head-movement produced motion parallax in simulated driving".

show abstract

“…They achieved a latency of only 200µs, though the memory limitations of their platform permitted parallax in only one dimension. The closest example to ours is that of Ng et al [9], who built a low latency direct touch interaction interface. They also designed theirs for latency interaction experiments.…”

Section: Related Workmentioning

confidence: 99%

“…FPGAs have been used for real-time rendering before, but with limited scope and tight vertical integration (e.g. [8], [9]). The dataflow computing paradigm and tools such as Maxeler's MaxCompiler are making it possible to construct image generators with entirely novel architectures, with similar complexity of programming as traditional GPUs [10].…”

Section: Introductionmentioning

confidence: 99%

Ultra low latency dataflow renderer

Friston

Steed

Tilbury

et al. 2015

2015 25th International Conference on Field Programmable Logic and Applications (FPL)

View full text Add to dashboard Cite

Abstract-In highly interactive applications, low latency (the time between a user's action, and the response to this action) is critical for a good user experience. Traditional GPU architectures can make very low latencies difficult to achieve. This is because they are designed first and foremost to implement the painter's algorithm -a rendering algorithm that trades-off visual realism for moderate computational speed and high scene dynamism. The dataflow programming paradigm, along with dedicated toolchains such as Maxeler's MaxCompiler, enable the design of application-specific graphics accelerators. Such accelerators, however, have the advantage that their architecture can be completely customised. In this paper we present a custom renderer that composites 2D sprites and maps to emulate a graphical user interface. It was designed to facilitate user interaction tests described in our previous work. Our design is ultra low latency, updating what is being driven to the display within 1 ms of receiving user input. This is far lower than traditional GPUs. We describe the operation of our renderer, and our novel DVI display driver output stage. We measure a latency of under 1 ms for our renderer, with an end-to-end delay of 6 ms for our whole apparatus. We compare this with the end-to-end latency of the same apparatus built with a modern GPU, which we measure at 20 ms.

show abstract

Designing for low-latency direct-touch input

Cited by 162 publications

References 22 publications

Using High Frequency Accelerometer and Mouse to Compensate for End-to-end Latency in Indirect Interaction

Using High Frequency Accelerometer and Mouse to Compensate for End-to-end Latency in Indirect Interaction

Driving in Virtual Reality: Investigations in Effects of Latency and Level of Virtuality

Ultra low latency dataflow renderer

Contact Info

Product

Resources

About