Eye Contact Correction using Deep Neural Networks

Isikdogan, Leo F.; Gerasimow, Timo; Michael, Gilad

doi:10.1109/wacv45572.2020.9093554

Cited by 5 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The gaze-angular offset effect can be compensated using various software-and hardware-based methods. [3] suggest a software-based technique of eye contact correction by redirecting gaze from an arbitrary angle to the center, alternating the video using a deep convolutional neural network. Another method [16] suggests changing the conversation members' gaze with 3D modeled eyeballs.…”

Section: State Of the Artmentioning

confidence: 99%

See 1 more Smart Citation

Subjective Influence of Camera-Gaze Angular Offset

et al. 2022

View full text Add to dashboard Cite

The study aimed to evaluate the difficulty in maintaining eye contact during a teleconference for different camera-gaze angular offsets. Videoconferencing systems may compromise the eye contact between participants due to imperfect angular alignment between the center of the screen and the camera. During a teleconference, difficulty maintaining eye contact may be perceived as uncomfortable or unsatisfying by participants in the call. An experiment deploying 33 test subjects was performed and evaluated. Groups of three test subjects performed a videoconference call. The effect of camera-gaze angular offset was quantified by assessing Effectiveness and Engagement ratings. It was shown that even 10 degrees of misalignment cause a statistically significant drop in Effectiveness for business-type teleconference calls. Camera-gaze angular offset is an important parameter to be considered during videoconferencing equipment design or setup. Higher misalignments may cause significant drops in perceived call quality (Quality of Experience). Our research results can be practical in conferencing equipment design and setup, modern telecommunication equipment design, workplace ergonomics, or distance learning equipment setup.

show abstract

Section: State Of the Artmentioning

confidence: 99%

“…Various software and hardware solutions are implemented to compensate for the gaze-angular offset [3]. To assess the human perception of this phenomenon, user QoE (Quality of Experience) tests may be used.…”

Section: Introductionmentioning

confidence: 99%

Subjective Influence of Camera-Gaze Angular Offset

et al. 2022

View full text Add to dashboard Cite

show abstract

“…49 A software solution to this problem has already been worked on, but, so far, it has had no effect on practical use. 50 Another visual feature of common software is that a miniature of one's own video is usually displayed on the screen, so that one can see not only one's counterpart but also oneself, as if in a small mirror. Studies show that this has significant effects on attention and communication behaviour.…”

Section: Feature Articlementioning

confidence: 99%

‘Can you hear me?’: communication, relationship and ethics in video-based telepsychiatric consultations

Frittgen¹,

Haltaufderheide

2021

J Med Ethics

View full text Add to dashboard Cite

Telepsychiatry has long been discussed as a supplement to or substitute for face-to-face therapeutic consultations. The current pandemic crisis has fueled the development in an unprecedented way. More and more psychiatric consultations are now carried out online as video-based consultations. Treatment results appear to be comparable with those of face-to-face care in terms of clinical outcome, acceptance, adherence and patient satisfaction. However, evidence on videoconferencing in a variety of different fields indicates that there are extensive changes in the communication behaviour in online conversations. We hypothesise that this might impact ethically relevant aspects of the therapeutic relationship, which plays a prominent role in psychiatry. In this paper, we review effects of video-based consultations on communication between therapists and patients in psychiatry. Based on a common understanding of video-based consultations as changing the lived experience of communication, we categorise these effects according to sensory, spatial and technical aspects. Departing from a power-based model of therapeutic relationships, we then discuss the ethical significance of this changed communication situation, based on dimensions of respect for autonomy, lucidity, fidelity, justice and humanity. We conclude that there is evidence for ethically relevant changes of the therapeutic relationship in video-based telepsychiatric consultations. These changes need to be more carefully considered in psychiatric practice and future studies.

show abstract

“…Without modifying hardware, researchers at Intel achieve gaze correction using an encoder-decoder network named ECC-Net (Isikdogan et al, 2020). The model is effective in its task of modifying a user image such that the eyes are directed toward the camera; it is therefore reasoned that the system intermediately and implicitly learns the gaze direction in the training process.…”

Section: Gaze Correctionmentioning

confidence: 99%

Restoring Eye Contact to the Virtual Classroom with Machine Learning

Greer,

Dubnov

2021

Preprint

View full text Add to dashboard Cite

Nonverbal communication, in particular eye contact, is a critical element of the music classroom, shown to keep students on task, coordinate musical flow, and communicate improvisational ideas. Unfortunately, this nonverbal aspect to performance and pedagogy is lost in the virtual classroom. In this paper, we propose a machine learning system which uses single instance, single camera image frames as input to estimate the gaze target of a user seated in front of their computer, augmenting the user's video feed with a display of the estimated gaze target and thereby restoring nonverbal communication of directed gaze. The proposed estimation system consists of modular machine learning blocks, leading to a target-oriented (rather than coordinateoriented) gaze prediction. We instantiate one such example of the complete system to run a pilot study in a virtual music classroom over Zoom software. Inference time and accuracy meet benchmarks for videoconferencing applications, and quantitative and qualitative results of pilot experiments include improved success of cue interpretation and student-reported formation of collaborative, communicative relationships between conductor and musician.

show abstract

Eye Contact Correction using Deep Neural Networks

Cited by 5 publications

References 24 publications

Subjective Influence of Camera-Gaze Angular Offset

Subjective Influence of Camera-Gaze Angular Offset

‘Can you hear me?’: communication, relationship and ethics in video-based telepsychiatric consultations

Restoring Eye Contact to the Virtual Classroom with Machine Learning

Contact Info

Product

Resources

About