Server-side object recognition and client-side object tracking for mobile augmented reality

Gammeter, Stephan; Gassmann, Alexander; Bossard, Lukas; Quack, Till; Gool, Luc Van

doi:10.1109/cvprw.2010.5543248

Cited by 82 publications

(37 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Unlike existing systems [20,14,22] that need to send every frame (or as many as the network can handle) to the server, Glimpse reduces bandwidth consumption by sending only certain trigger frames to the server to obtain the object recognition hints.…”

Section: Trigger Framesmentioning

confidence: 99%

“…For example, some papers [25,27] deploy a recognition system that allows clients to upload images to the server for object classification. Other papers [20,14] provide continuous object recognition on mobile devices by running the tracker on the mobile client, and the recognition on a server. Another paper [58] proposes a location-aware face recognition framework that uses location information as a hint to reduce the search space of the recognition algorithm.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Glimpse

Chen

Ravindranath

Deng

et al. 2015

Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems

279

View full text Add to dashboard Cite

Glimpse is a continuous, real-time object recognition system for camera-equipped mobile devices. Glimpse captures full-motion video, locates objects of interest, recognizes and labels them, and tracks them from frame to frame for the user. Because the algorithms for object recognition entail significant computation, Glimpse runs them on server machines. When the latency between the server and mobile device is higher than a frame-time, this approach lowers objectrecognition accuracy. To regain accuracy, Glimpse uses an active cache of video frames on the mobile device. A subset of the frames in the active cache are used to track objects on the mobile, using (stale) hints about objects that arrive from the server from time to time. To reduce network bandwidth usage, Glimpse computes trigger frames to send to the server for recognizing and labeling. Experiments with Android smartphones and Google Glass over Verizon, AT&T, and a campus Wi-Fi network show that with hardware face detection support (available on many mobile devices), Glimpse achieves precision between 96.4% to 99.8% for continuous face recognition, which improves over a scheme performing hardware face detection and server-side recognition without Glimpse's techniques by between 1.8-2.5×. The improvement in precision for face recognition without hardware detection is between 1.6-5.5×. For road sign recognition, which does not have a hardware detector, Glimpse achieves precision between 75% and 80%; without Glimpse, continuous detection is non-functional (0.2%-1.9% precision). Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. Categories and Subject Descriptors

show abstract

Section: Trigger Framesmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Glimpse

Chen

Ravindranath

Deng

et al. 2015

Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems

279

View full text Add to dashboard Cite

show abstract

“…Many previous works using a server-client architecture for tracking are focused on 2D object recognition [24,9] or are limited to homography-based tracking of flat objects [10]. Our system supports full 6DOF visual tracking of the entire surrounding enviroment.…”

Section: Related Workmentioning

confidence: 99%

Wide-area scene mapping for mobile visual tracking

Ventura

Höllerer

2012

2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

View full text Add to dashboard Cite

We propose a system for easily preparing arbitrary wide-area environments for subsequent real-time tracking with a handheld device. Our system evaluation shows that minimal user effort is required to initialize a camera tracking session in an unprepared environment. We combine panoramas captured using a handheld omnidirectional camera from several viewpoints to create a point cloud model. After the offline modeling step, live camera pose tracking is initialized by feature point matching, and continuously updated by aligning the point cloud model to the camera image. Given a reconstruction made with less than five minutes of video, we achieve below 25 cm translational error and 0.5 degrees rotational error for over 80% of images tested. In contrast to camera-based simultaneous localization and mapping (SLAM) systems, our methods are suitable for handheld use in large outdoor spaces.

show abstract

“…Examples of desirable augmented reality (AR) applications for mobile devices include e-Health [49], tourist assistance [26,27,61], gaming, navigation and industrial maintenance [53]. AR is defined as "a real-time direct or indirect view of the real world environment that has been enhanced/augmented by adding virtual computer generated information to it" [18].…”

Section: Tourist Assistance Applicationmentioning

confidence: 99%

A Proxy-Based Mobile Computing Infrastructure

Khalaj

Lutfiyya

Perry

2012

2012 Third FTRA International Conference on Mobile, Ubiquitous, and Intelligent Computing

View full text Add to dashboard Cite

In recent years, there has been a huge growth in mobile applications. More mobile users are able to access Internet services via their mobile devices e.g., smartphones and tablets. Some of these applications are highly interactive and resource intensive. Mobile applications, with limited storage capacity, slow processors and limited battery life, could be connected to the remote servers in clouds for leveraging resources. For example, weather applications use a remote service that collects weather data and make this data available through a well-defined API. This represents a static partitioning of functionality between mobile devices and a remote server that is determined at run-time.Regardless of the network distance between the cloud infrastructure and the mobile device, the use of a remote service is well suited for mobile device applications with relatively little data to be transferred. However, long distances between a mobile device and remote services make this approach unsuitable for applications that require larger amounts of data to be transferred and/or have a high level of interactiveness with the user This includes mobile video communications (e.g., Skype, Face-Time, Google-Hangout), gaming applications that require sophisticated rendering and cloud media analysis that can be used to offer more personalized services. The latency incurred with this architecture makes it difficult to support real-time and interactive applications. A related problem is that the static partitioning strategy is not always suitable for all network conditions and inputs. For example, let us consider a speech recognition application. The performance depends on the size of the input and the type of connectivity to the backbone. Another challenge is that the communication medium between the mobile application and the remote service includes wireless links. Wireless links are more error prone and have less bandwidth than wired links. Often a mobile application may be disconnected.One approach to addressing these challenges is the use of a proxy. A proxy is computing power that is located at the network edge. This allows it to address problems with latency. It is possible for a proxy to have services that allow for offloading tasks from either the cloud or the mobile device and to deal with communication challenges between the mobile application and the mobile device.This work proposes a proxy-based system that acts as a middleware between the mobile application and the remote service. The proposed middleware consists of a set of proxies that provide services. The proposed middleware includes services for proxy discovery and selection, mechanisms for dealing with balancing loads on proxies and handoff. A prototype was developed to assess the effectiveness of the proposed proxy-based system.

show abstract

Server-side object recognition and client-side object tracking for mobile augmented reality

Cited by 82 publications

References 22 publications

Glimpse

Glimpse

Wide-area scene mapping for mobile visual tracking

A Proxy-Based Mobile Computing Infrastructure

Contact Info

Product

Resources

About