e2-Mask: Design and Implementation of a Mask-Type Display to Support Face-to-Face Communication

Abstract

In Face-to-Face communication, his or her face is an important part that decides the impression of a person because a person estimates the personality of the interlocutor from the features of the face of the interlocutor. Therefore, we made a hypothesis own impression and personality of giving to others by changing the own face freely. Then we proposed the e2-Mask that a mask-type display that synchronizes with facial expressions and substitutes an avatar for a human face.

Face mask design to mitigate facial expression occlusion

“…Utilising electrochromic (EC) display technology we were able to design and construct smart face masks that present a believable step towards a productizable solution. Whilst we did not explore mechanisms to control the display switching we believe that straightforward solutions can be found based on simplified versions of prior work on the topic [7,18]. Our approach of evaluating the face masks over a video conversation was mandated by the social isolation rules in operation at the time of the study.…”

“…Focusing on face masks covering the mouth, Kumazaki et al [7] developed a face mask display that tracks the wearer's facial expression and presents a photo realistic representation on the outward facing display Focusing to sensing the facial expressions of the wearer, Umezawa et al [18] developed the e2-Mask, that utilized 40 photo reflective sensors on the face to recreate facial expressions in an avatar. To support telepresence, Misawa and Rekimoto [11] demonstrated the ChameleonMask that displays a remote user's face on the face of a human-surrogate, and concluded that the surrogate is regarded as the actual (remote) person.…”

“…As prior work has already demonstrated methods to sense facial expressions [7,18], our work focused only on exploring the the display side of the interaction, and hence relied on manual switching of the display state by the wearer, via a wired switch. This approach was suitable for our chosen Wizard of Oz evaluation method [9], where the face masks were worn by the test moderator.…”

Digital Full-Face Mask Display with Expression Recognition using Embedded Photo Reflective Sensor Arrays

“…We have previously explored a camera-based digital mask display, e2-Mask [37], which used two flat tablet displays and two webcameras to capture and reflect the face of the wearer on an avatar (Figure 3). Both displays and cameras are mounted on each side of the mask device, i.e.…”

“…To evaluate the feasibility of a digital face mask display based on photo-reflector sensor arrays (e2-MaskZ), we conducted user experiments to compare the sense of self-agency and self-ownership [9] toward the controlled avatar, between our mask and the camera-based digital face mask display (e2-Mask) [37].…”