Face typing: Vision-based perceptual interface for hands-free text entry with a scrollable virtual keyboard

Gizatdinova, Yulia; Špakov, Oleg; Surakka, Veikko

doi:10.1109/wacv.2012.6162997

Cited by 20 publications

(13 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to the techniques using gaze and voice, numerous interfaces using face gesture [19], nose [20], head direction [21], foot [22], and EMG [23], [24] have been developed. However, these techniques are difficult to employ in various applications due to the limited environment and usability limitations.…”

Section: A Input Technologies As a Hands-free Interfacementioning

confidence: 99%

The Study of a Classification Technique for Numeric Gaze-Writing Entry in Hands-Free Interface

2019

View full text Add to dashboard Cite

Recently, many applications are developed in numerous domains with various environments. Since some environments require hands-free applications, new technology is needed for the input interfaces other than the mouse and keyboard. Therefore, to meet the needs, many researchers have begun to investigate the gaze and voice for the input technology. In particular, there are many approaches to render virtual keyboards with the gaze. However, since the virtual keyboards hide the screen space, this technique can only be applied in limited environments. In this paper, we propose a classification technique for gaze-written numbers as the hands-free interface. Since the gaze-writing is less accurate compared to the virtual keyboard typing, we apply the convolutional neural network (CNN) deep learning algorithm to recognize the gaze-writing and improve the classification accuracy. Besides, we create new gaze-writing datasets for training, gaze MNIST (gMNIST), by modifying the MNIST data with features of the gaze movement patterns. For the evaluation, we compare our approach with the basic CNN structures using the original MNIST dataset. Our study will allow us to have more options for the input interfaces and expand our choices in hands-free environments. INDEX TERMS Gaze-writing, input technique, MNIST, Eye tracking, machine learning.

show abstract

Section: A Input Technologies As a Hands-free Interfacementioning

confidence: 99%

The Study of a Classification Technique for Numeric Gaze-Writing Entry in Hands-Free Interface

2019

View full text Add to dashboard Cite

show abstract

“…Text typing in such environment, as it was mentioned before, is usually offered through on-screen keyboard operated by pointing mechanism or traverse procedure. A few examples can be found in literature: [14,26] and [33]. Some solutions have also been shared publicly as open source projects [1] and [28].…”

Section: Head Operated Interfacesmentioning

confidence: 99%

“…16 The results of the experiments in the visible spectrum reaches 16.95 while disabled users managed to complete the task with the mean value of 37.12 seconds [33] which is 119% longer). Some typing speeds of vision-based interfaces are compared in [14] where the authors reference works with 25, 31 and 44 cpm for the camera mouse approach and pointing procedure. As it was mentioned before, the pointing procedure requires substantial precision and may be not adequate for some users.…”

Section: Interface Evaluationmentioning

confidence: 99%

“…As it was mentioned before, the pointing procedure requires substantial precision and may be not adequate for some users. A letter-scanning (traverse) procedure [14] used in the interface proposed by Grauman et al allows users to achieve 5.7 cpm.…”

Section: Interface Evaluationmentioning

confidence: 99%

“…clicking events), different non-rigid motions are employed. Some of the examples include: eye blinks [24,30,37], mouth shape changes (opening, closing or stretching) [5,14,33,36], brows movements [14] or cheeks twitch.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Touchless typing with head movements captured in thermal spectrum

Nowosielski

Forczmański

2018

Pattern Anal Applic

View full text Add to dashboard Cite

Many physically challenged people are unable to operate standard electronic equipment or computer input devices. They need special assistive technologies and one of the options is the head operated interface. Face-oriented algorithms often assume a particular level of lighting with adequate intensity and spatial configuration. In the paper, we propose a thermalimaging-based algorithm of head operating typing. It does not assume the visible light illumination. We investigated, in context of thermal imagery, several contemporary general purpose object detectors known to be accurate in case of images captured by the visible light camera. Then, a selected face detector is employed in the head operated interface analysing head movements in the thermal spectrum. The attention has been focused on the problem of touchless typing which is performed in the existing solutions either through the camera mouse or through traverse procedure with an addition mechanism (like eye blink or mouth open) needed for clicking events in both cases. Our novel solution for touchless typing with head movements combines the thermal imaging for capturing user action with the hierarchical letter selection procedure. The solution employed allows to reach any alphabet character in just three steps, i.e. with directional head movements, without the need of any additional mechanisms for clicking events.

show abstract

Text Entry by Rotary Head Movements

Nowosielski

2018

Image Processing and Communications Challenges 10

View full text Add to dashboard Cite

Face typing: Vision-based perceptual interface for hands-free text entry with a scrollable virtual keyboard

Cited by 20 publications

References 18 publications

The Study of a Classification Technique for Numeric Gaze-Writing Entry in Hands-Free Interface

The Study of a Classification Technique for Numeric Gaze-Writing Entry in Hands-Free Interface

Touchless typing with head movements captured in thermal spectrum

Text Entry by Rotary Head Movements

Contact Info

Product

Resources

About