TextSLAM: Visual SLAM with Planar Text Features

Li, Boying; Zou, Danping; Sartori, Daniele; Pei, Ling; Wang, Yu

doi:10.1109/icra40945.2020.9197233

Cited by 32 publications

(16 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…PoseNet [55] was one of the earliest works that uses CNN to estimate camera pose in an end-to-end manner. After that, many studies have sprung up [10], [19], [56], [57]. Deep VO [58] use RNN to model the relationship of motion dynamic and image sequences.…”

Section: B Slammentioning

confidence: 99%

SBAS: Salient Bundle Adjustment for Visual SLAM

Wang

Ren

et al. 2021

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

Recently, the philosophy of visual saliency and attention has started to gain popularity in the robotics community. Therefore, this paper aims to mimic this mechanism in SLAM framework by using saliency prediction model. Comparing with traditional SLAM that treated all feature points as equal important in optimization process, we think that the salient feature points should play more important role in optimization process. Therefore, we proposed a saliency model to predict the saliency map, which can capture both scene semantic and geometric information. Then, we proposed Salient Bundle Adjustment by using the value of saliency map as the weight of the feature points in traditional Bundle Adjustment approach. Exhaustive experiments conducted with the state-of-the-art algorithm in KITTI and EuRoc datasets show that our proposed algorithm outperforms existing algorithms in both indoor and outdoor environments. Finally, we will make our saliency dataset and relevant source code open-source for enabling future research.

show abstract

Section: B Slammentioning

confidence: 99%

SBAS: Salient Bundle Adjustment for Visual SLAM

Wang

Ren

et al. 2021

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

show abstract

“…Contextual features in the environment have been used for a variety of purposes, including: (1) localization [36][37][38], (2) map annotation [21,22,[39][40][41][42][43], (3) SLAM using text [44,45], and (4) Semantic SLAM [32,[46][47][48][49].…”

Section: Mapping and Localization Using Contextual Informationmentioning

confidence: 99%

“…In [44], OCR was used in SLAM by fusing measurements to the centroids of unique text instances with visual inertial odometry via incremental Smoothing and Mapping. In [45], SLAM was performed using the planar features of text instances in the environment by minimizing the projected photometric error of detected text boxes via bundle adjustment.…”

Section: Slam Using Text Featuresmentioning

confidence: 99%

“…Using text, on the other hand, results in a more accurate classification of products since all products in the environment have a text label, either on the shelf or the packaging themselves. In [45], text features were used for SLAM, however, only the planar surfaces of the detected text were utilized and the method did not annotate the map with the contextual information of the text strings found [45]. Therefore, these approaches cannot be used for our grocery search problem as: (1) real grocery environments contain repeated features (e.g., same text on signage, shelves, posters, etc.…”

Section: Semantic Slammentioning

confidence: 99%

See 1 more Smart Citation

A Robot Architecture Using ContextSLAM to Find Products in Unknown Crowded Retail Environments

et al. 2021

View full text Add to dashboard Cite

Grocery shoppers must negotiate cluttered, crowded, and complex store layouts containing a vast variety of products to make their intended purchases. This complexity may prevent even experienced shoppers from finding their grocery items, consuming a lot of their time and resulting in monetary loss for the store. To address these issues, we present a generic grocery robot architecture for the autonomous search and localization of products in crowded dynamic unknown grocery store environments using a unique context Simultaneous Localization and Mapping (contextSLAM) method. The contextSLAM method uniquely creates contextually rich maps through the online fusion of optical character recognition and occupancy grid information to locate products and aid in robot localization in an environment. The novelty of our robot architecture is in its ability to intelligently use geometric and contextual information within the context map to direct robot exploration in order to localize products in unknown environments in the presence of dynamic people. Extensive experiments were conducted with a mobile robot to validate the overall architecture and contextSLAM, including in a real grocery store. The results of the experiments showed that our architecture was capable of searching for and localizing all products in various grocery lists in different unknown environments.

show abstract

“…The first step is plane fitting for 3D points within the planar region. We obtain the plane parameters θ = −n/d ∈ R 3 as previous work [28,50] by solving the least squares problem…”

Section: Co-planar Constraintmentioning

confidence: 99%

StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation

Li¹,

Huang²,

Liu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Self-supervised monocular depth estimation has achieved impressive performance on outdoor datasets. Its performance however degrades notably in indoor environments because of the lack of textures. Without rich textures, the photometric consistency is too weak to train a good depth network. Inspired by the early works on indoor modeling, we leverage the structural regularities exhibited in indoor scenes, to train a better depth network. Specifically, we adopt two extra supervisory signals for self-supervised training: 1) the Manhattan normal constraint and 2) the co-planar constraint. The Manhattan normal constraint enforces the major surfaces (the floor, ceiling, and walls) to be aligned with dominant directions. The co-planar constraint states that the 3D points be well fitted by a plane if they are located within the same planar region. To generate the supervisory signals, we adopt two components to classify the major surface normal into dominant directions and detect the planar regions on the fly during training. As the predicted depth becomes more accurate after more training epochs, the supervisory signals also improve and in turn feedback to obtain a better depth model. Through extensive experiments on indoor benchmark datasets, the results show that our network outperforms the state-ofthe-art methods. The source code is available at https: //github.com/SJTU-ViSYS/StructDepth.

show abstract

TextSLAM: Visual SLAM with Planar Text Features

Cited by 32 publications

References 35 publications

SBAS: Salient Bundle Adjustment for Visual SLAM

SBAS: Salient Bundle Adjustment for Visual SLAM

A Robot Architecture Using ContextSLAM to Find Products in Unknown Crowded Retail Environments

StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation

Contact Info

Product

Resources

About