The goal of this research is to explore new interaction metaphors for augmented reality on mobile phones, i.e. applications where users look at the live image of the device's video camera and 3D virtual objects enrich the scene that they see. Common interaction concepts for such applications are often limited to pure 2D pointing and clicking on the device's touch screen. Such an interaction with virtual objects is not only restrictive but also difficult, for example, due to the small form factor. In this article, we investigate the potential of finger tracking for gesture-based interaction. We present two experiments evaluating canonical operations such as translation, rotation, and scaling of virtual objects with respect to performance (time and accuracy) and engagement (subjective user feedback). Our results indicate a high entertainment value, but low accuracy if objects are manipulated in midair, suggesting great possibilities for leisure applications but limited usage for serious tasks.

The Lifelog Search Challenge (LSC) is an international content retrieval competition that evaluates search for personal lifelog data. At the LSC, content-based search is performed over a multi-modal dataset, continuously recorded by a lifelogger over 27 days, consisting of multimedia content, biometric data, human activity data, and information activities data. In this work, we report on the first LSC that took place in Yokohama, Japan in 2018 as a special workshop at ACM International Conference on Multimedia Retrieval 2018 (ICMR 2018). We describe the general idea of this challenge, summarise the participating search systems as well as the evaluation procedure, and analyse the search performance of the teams in various aspects. We try to identify reasons why some systems performed better than others and provide an outlook as well as open issues for upcoming iterations of the challenge.

The Lifelog Search Challenge (LSC) is an annual comparative benchmarking activity for comparing approaches to interactive retrieval from multi-modal lifelogs. LSC'20, the third such challenge, attracts fourteen participants with their interactive lifelog retrieval systems. These systems are comparatively evaluated in front of a live-audience at the LSC workshop at ACM ICMR'20 in Dublin, Ireland. This overview motivates the challenge, presents the dataset and system configuration used in the challenge, and briefly presents the participating teams. CCS CONCEPTS • Human-centered computing → Empirical studies in interaction design; • Information systems → Mobile information processing systems; Search interfaces.

We present recent w ork on improving the performance of automated speech recognizers by using additional visual information Lip-Speechreading, achieving error reduction of up to 50. This paper focuses on di erent methods of combining the visual and acoustic data to improve the recognition performance. We show this on an extension of an existing state-of-the-art speech recognition system, a modular MS-TDNN. We have developed adaptive combination methods at several levels of the recognition network. Additional information such as estimated signal-to-noise ratio SNR is used in some cases. The results of the di erent combination methods are shown for clean speech and data with arti cial noise white, music, motor. The new combination methods adapt automatically to varying noise conditions making hand-tuned parameters unnecessary.

Today, videos can be replayed on modern handheld devices, such as multimedia cellphones and personal digital assistants (PDAs), due to significant improvements in their processing power. However, screen size remains a limiting resource making it hard, if not impossible to adapt common approaches for video browsing to such mobile devices. In this paper we propose a new interface for the pen-based navigation of videos on PDAs and multimedia cellphones. Our solution -called the MobileZoomSlider -enables users to intuitively skim a video along the timeline on different granularity levels. In addition, it allows for continuous manipulation of replay speed for browsing purposes. Both interaction concepts are seamlessly integrated into the overall interface, thus taking optimum advantage of the limited screen space. Our claims are verified with a first evaluation which proves the suitability of the overall concept.