TeethTap: Recognizing Discrete Teeth Gestures Using Motion and Acoustic Sensing on an Earpiece

Sun, Wei; Li, Franklin Mingzhe; Steeper, Benjamin; Xu, Songlin; Tian, Feng; Zhang, Cheng

doi:10.1145/3397481.3450645

Cited by 32 publications

(9 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, when interacting with a map view to find the best local dining option, a user may frequently pan around and zoom (in and out) to view different restaurants, and both the duration of stay on a particular restaurant and how many times it is viewed back and forth could be leveraged to approximate the user's interest and investment of effort. One way to address these concerns is to leverage a more diverse set of behavioral signals and potentially signal combinations, such as scrolling, mouse panning, zooming, eye tracking [35,36,89,90], and facial gestures tracking [70,117] to collect a more accurate picture of what users are seeing on screen. Another future direction that could be fruitful is to take a machine learning approach instead of the current rule-based approach for approximating content importance using behavioral signals.…”

Section: Future Workmentioning

confidence: 99%

Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making

Liu,

Kittur,

Myers

2022

Preprint

View full text Add to dashboard Cite

Developers perform online sensemaking on a daily basis, such as researching and choosing libraries and APIs. Prior research has introduced tools that help developers capture information from various sources and organize it into structures useful for subsequent decision-making. However, it remains a laborious process for developers to manually identify and clip content, maintaining its provenance and synthesizing it with other content. In this work, we introduce a new system called Crystalline that automatically collects and organizes information into tabular structures as the user searches and browses the web. It leverages natural language processing to automatically group similar criteria together to reduce clutter, and uses passive behavioral signals such as mouse movement and dwell time to infer what information to collect and how to visualize and prioritize it. Our user study suggests that developers are able to create comparison tables about 20% faster with a 60% reduction in operational cost without sacrificing the quality of the tables. CCS CONCEPTS• Information systems → Decision support systems; • Software and its engineering → Software design tradeoffs; • Humancentered computing → Graphical user interfaces.

show abstract

Section: Future Workmentioning

confidence: 99%

Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making

Liu,

Kittur,

Myers

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…4). One direct solution to make lips more visible to the DHH viewers is to have the speaker equip a wearable camera [31,40] or explore earable sensing systems [66] that track the lip movement. However, such an approach would be very cumbersome and costly to individual content creators.…”

Section: 51mentioning

confidence: 99%

An Exploration of Captioning Practices and Challenges of Individual Content Creators on YouTube for People with Hearing Impairments

Li,

Lu,

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Fig. 1. YouTubers' and Deaf and Hard-of-Hearing audiences' complaints and campaigns on asking for better captions of videos created by individual content creators on video-sharing platforms (e.g., #NoMoreCraptions). Deaf and Hard-of-Hearing (DHH) audiences have long complained about caption qualities for many online videos created by individual content creators on video-sharing platforms (e.g., YouTube). However, there lack explorations of practices, challenges, and perceptions of online video captions from the perspectives of both individual content creators and DHH audiences. In this work, we first explore DHH audiences' feedback on and reactions to YouTube video captions through interviews with 13 DHH individuals, and uncover DHH audiences' experiences, challenges, and perceptions on watching videos created by individual content creators (e.g., manually added caption tags could create additional confidence and trust in caption qualities for DHH audiences). We then discover individual content creators' practices, challenges, and perceptions on captioning their videos (e.g., back-captioning problems) by conducting a YouTube video analysis with 189 captioning-related YouTube videos, followed by a survey with 62 individual content creators. Overall, our findings provide an in-depth understanding of captions generated by individual content creators and bridge the knowledge gap mutually between content creators and DHH audiences on captions. CCS Concepts: • Human-centered computing → Empirical studies in accessibility.

show abstract

“…Eye gesture input is the most frequently used method for hands-free input such as eye gesture recognition [31][32][33] and gaze interaction (Orbits [6], GazeTap [15]). And mouth-related interface (Whoosh [38], TieLent [20]) such as tongue interface [11,24] and teeth interface (TeethTap [43],Bitey [2], EarSense [37]) also offers users a novel way for hands-free input. What's more, ear-based interaction (EarRumble [39]), waist gestures (HulaMove [48]) and foot gestures (FootUI [17], FEETICHE [26]) are also explored by researchers.…”

Section: Hands-free Gesture Inputmentioning

confidence: 99%

HeadText: Exploring Hands-free Text Entry using Head Gestures by Motion Sensing on a Smart Earpiece

Xu¹,

Wang²,

Fang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Fig. 1. Application scenarios of HeadText. Fig.a shows that users could perform head gestures to respond to messages when they are eating and their hands are holding the cake. Fig. b presents that a user could use HeadText to input text on a smart watch, whose text entry is originally limited due to the small screen size. Fig.c shows that HeadText could be utilized to help users with motor impairments for text entry. Fig. d presents the application scenario where users could perform head gestures for text entry on the smart glass which is private and socially acceptable in public. We present HeadText, a hands-free technique on a smart earpiece for text entry by motion sensing. Users input text utilizing only head gestures for key selection, word selection, word commitment and word cancelling tasks. Head gesture recognition is supported by motion sensing on a smart earpiece to capture head moving signals and machine learning algorithms (K-Nearest-Neighbor (KNN)with a Dynamic Time Warping (DTW) distance measurement). A 10-participant user study proved that HeadText could recognize head gestures at an accuracy of 94.29%. After that, the second user study presented that HeadText could achieve a maximum accuracy

show abstract

TeethTap: Recognizing Discrete Teeth Gestures Using Motion and Acoustic Sensing on an Earpiece

Cited by 32 publications

References 33 publications

Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making

Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making

An Exploration of Captioning Practices and Challenges of Individual Content Creators on YouTube for People with Hearing Impairments

HeadText: Exploring Hands-free Text Entry using Head Gestures by Motion Sensing on a Smart Earpiece

Contact Info

Product

Resources

About