Efficient mobile AR technology using scalable recognition and tracking based on server-client model

Jung, Jaeho; Ha, Jae‐Won; Lee, Sangwook; Rojas, Francisco Arturo; Yang, Hyun Seung

doi:10.1016/j.cag.2012.01.004

Cited by 19 publications

(12 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system architecture that consists of a mobile device client and the cloud server as following is a short description of the overall procedure: On the client-side, the server takes the image, compresses it and saves its size and pixels distribution the object recognition CNN in the server extracts feature points within the view once the image is received. Unlike [8], our software does not require the client to enter the area of interest for the recognition to be achieved. Then the result is direct gets sent back to the mobile application, including the label information of the target object.…”

Section: Cloud-based Object Recognition Servermentioning

confidence: 99%

Cloud-Based ImageNet Object Recognition For Mobile Devices

Saeed¹,

Schonfeld²

2020

Proceedings of the Proceedings of the 1st International Multi-Disciplinary Conference Theme: Sustainable Development and Smart

View full text Add to dashboard Cite

User reliance on real-time applications is continuously increasing as the use of smartphone devices has tremendously increased in day to day life within the past few years. Smartphone devices computation power has significantly increased as well, however, there are still some scalability, performance challenges and some complications with real-time application such as limited computation capabilities and battery life of mobile devices. In this paper, we propose a cloud-based object recognition through task offloading to a high-speed server. We explore this design extensively and demonstrate a real-time solution for object recognition framework on mobile devices using a Convolution neural network (CNN) leveraging the ImageNet dataset and by optimizing the offloading process to minimize the time and energy needed. This framework use the emerging the Android operating system as a platform to connect with an object recognition server where the CNN deep learning resides and process received images. Using this method will overcome the design limited capacity of mobile devices since object recognition algorithms require high-speed calculations.

show abstract

Section: Cloud-based Object Recognition Servermentioning

confidence: 99%

Cloud-Based ImageNet Object Recognition For Mobile Devices

Saeed¹,

Schonfeld²

2020

Proceedings of the Proceedings of the 1st International Multi-Disciplinary Conference Theme: Sustainable Development and Smart

View full text Add to dashboard Cite

show abstract

“…The recognition result includes the initial posture of the object, and the client tracks the object using the recognition result. In [16], Jung et al implemented a server-client-based large-scale mobile AR system for 10,000 objects using vocabulary trees [17] as a recognizer. Because it is impossible to transmit all the real-time images from the client to the server, only the frame desired by the user is transmitted to the server.…”

Section: Related Workmentioning

confidence: 99%

Real-time Object Recognition with Pose Initialization for Large-scale Standalone Mobile Augmented Reality

Lee¹

2020

KSII TIIS

View full text Add to dashboard Cite

Mobile devices such as smartphones are very attractive targets for augmented reality (AR) services, but their limited resources make it difficult to increase the number of objects to be recognized. When the recognition process is scaled to a large number of objects, it typically requires significant computation time and memory. Therefore, most large-scale mobile AR systems rely on a server to outsource recognition process to a high-performance PC, but this limits the scenarios available in the AR services. As a part of realizing large-scale standalone mobile AR, this paper presents a solution to the problem of accuracy, memory, and speed for large-scale object recognition. To this end, we design our own basic feature and realize spatial locality, selective feature extraction, rough pose estimation, and selective feature matching. Experiments are performed to verify the appropriateness of the proposed method for realizing large-scale standalone mobile AR in terms of efficiency and accuracy.

show abstract

“…As far as we concern, SIFTGPU is the most mature and fastest SIFT implementation on GPU before our CLSIFT. With great reputations, SIFTGPU is widely integrated into related systems like image searching engine [12], 3D modeling [13], augmented reality [14] and so on. So We have compared our CLSIFT with the famous SIFTGPU latest version SIFTGPU V400 to verify our introduced techniques' efficiency.…”

Section: Related Workmentioning

confidence: 99%

CLSIFT: An Optimization Study of the Scale Invariance Feature Transform on GPUs

Wang

Zhang

Guoping³

et al. 2013

2013 IEEE 10th International Conference on High Performance Computing and Communications &Amp; 2013 IEEE International Conferen

View full text Add to dashboard Cite

Scale Invariance Feature Transform (SIFT) is quite suitable for image matching because of its invariance to image scaling, rotation and slight changes in illumination or viewpoint. However, due to high computation complexity it's technically challenging to deploy SIFT in real time application situations. To address this problem, we propose CLSIFT, an OpenCL based highly speeded up and performance portable SIFT solution. Important optimization techniques employed in CLSIFT such as: (1) For less global memory traffic, independent logical functions are merged into the same kernel to reuse data.(2) loop buffers are introduced in for data and intermediate results reusing.(3)Task queue used to schedule threads in the same branch to remove branch divergences. (4) Data partition is based on the statics patterns for workload balance among workgroups. (5) Overlap of CPU time and better parallel strategies are used too. With all mentioned efforts, CLSIFT processes lena.jpg at 74.2 FPS and 43.4FPS respectively on NVidia and AMD GPUS, much higher than CPU's nearly 10 FPS and the known fastest SIFTGPU's 39.8 FPS and 13FPS. Moreover in a quantitative comparison approach we analyze those successful strategies beating SIFTGPU, a famous existing GPU implementation. Additionally, we observe and conclude that NVidia GPU achieves better occupancy and performance due to some factors. Finally, we summarize some techniques and empirical guiding principles that may be shared by other applications on GPU.

show abstract

Efficient mobile AR technology using scalable recognition and tracking based on server-client model

Cited by 19 publications

References 12 publications

Cloud-Based ImageNet Object Recognition For Mobile Devices

Cloud-Based ImageNet Object Recognition For Mobile Devices

Real-time Object Recognition with Pose Initialization for Large-scale Standalone Mobile Augmented Reality

CLSIFT: An Optimization Study of the Scale Invariance Feature Transform on GPUs

Contact Info

Product

Resources

About