From Marker to Markerless in Augmented Reality

Oufqir, Zainab; Abderrahmani, Abdellatif El; Satori, Khalid

doi:10.1007/978-981-15-0947-6_57

Cited by 24 publications

(15 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning is a ground-breaking approach to the use of AI [16,27]. Eliminating markers is still a current topic and there is no single, optimal solution in the literature regarding this issue [28][29][30]. It is expected that the use of AI will eliminate the need to place physical markers, which will lead to full, automatic recognition of the environment and selected elements.…”

Section: Literature Reviewmentioning

confidence: 99%

Application of the Deep CNN-Based Method in Industrial System for Wire Marking Identification

et al. 2021

View full text Add to dashboard Cite

Industry 4.0, a term invented by Wolfgang Wahlster in Germany, is celebrating its 10th anniversary in 2021. Still, the digitalization of the production environment is one of the hottest topics in the computer science departments at universities and companies. Optimization of production processes or redefinition of the production concepts is meaningful in light of the current industrial and research agendas. Both the mentioned optimization and redefinition are considered in numerous subtopics and technologies. One of the most significant topics in these areas is the newest findings and applications of artificial intelligence (AI)—machine learning (ML) and deep convolutional neural networks (DCNNs). The authors invented a method and device that supports the wiring assembly in the control cabinet production process, namely, the Wire Label Reader (WLR) industrial system. The implementation of this device was a big technical challenge. It required very advanced IT technologies, ML, image recognition, and DCNN as well. This paper focuses on an in-depth description of the underlying methodology of this device, its construction, and foremostly, the assembly industrial processes, through which this device is implemented. It was significant for the authors to validate the usability of the device within mentioned production processes and to express both advantages and challenges connected to such assembly process development. The authors noted that in-depth studies connected to the effects of AI applications in the presented area are sparse. Further, the idea of the WLR device is presented while also including results of DCNN training (with recognition results of 99.7% although challenging conditions), the device implementation in the wire assembly production process, and its users’ opinions. The authors have analyzed how the WLR affects assembly process time and energy consumption, and accordingly, the advantages and challenges of the device. Among the most impressive results of the WLR implementation in the assembly process one can be mentioned—the device ensures significant process time reduction regardless of the number of characters printed on a wire.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Application of the Deep CNN-Based Method in Industrial System for Wire Marking Identification

et al. 2021

View full text Add to dashboard Cite

show abstract

“…This issue is not really relevant in some AR applications, in which no physical constraints are considered and virtual objects can be placed anywhere, even floating in the air. MR goes one step further, both worlds must be correctly overlapped considering a spatial consistency [8]. This implies the understanding of the user environment and a optimal adjustment in order to locate each virtual model, which represents every indoor facility, where it has been fixed considering both for the design and construction phase and maintenance phase.…”

Section: Introductionmentioning

confidence: 99%

“…Our markerless-based approach implies a significant advance for the use of mixed reality using geometrical data scanned from the user environment in order to know both the user location and orientation in indoor scenarios [8]. In this regard, there have been previous works focused on the removal of physical markers but with the drawback that additional sensing such as radio frequency identification, laser pointing and motion tracking must be added [22].…”

Section: Introductionmentioning

confidence: 99%

GEUINF: Real-Time Visualization of Indoor Facilities Using Mixed Reality

Jurado

Ortega

et al. 2021

Sensors

View full text Add to dashboard Cite

Mixed reality (MR) enables a novel way to visualize virtual objects on real scenarios considering physical constraints. This technology arises with other significant advances in the field of sensors fusion for human-centric 3D capturing. Recent advances for scanning the user environment, real-time visualization and 3D vision using ubiquitous systems like smartphones allow us to capture 3D data from the real world. In this paper, a disruptive application for assessing the status of indoor infrastructures is proposed. The installation and maintenance of hidden facilities such as water pipes, electrical lines and air conditioning tubes, which are usually occluded behind the wall, supposes tedious and inefficient tasks. Most of these infrastructures are digitized but they cannot be visualized onsite. In this research, we focused on the development of a new application (GEUINF) to be launched on smartphones that are capable of capturing 3D data of the real world by depth sensing. This information is relevant to determine the user position and orientation. Although previous approaches used fixed markers for this purpose, our application enables the estimation of both parameters with a centimeter accuracy without them. This novelty is possible since our method is based on a matching process between reconstructed walls of the real world and 3D planes of the replicated world in a virtual environment. Our markerless approach is based on scanning planar surfaces of the user environment and then, these are geometrically aligned with their corresponding virtual 3D entities. In a preprocessing phase, the 2D CAD geometry available from an architectural project is used to generate 3D models of an indoor building structure. In real time, these virtual elements are tracked with the real ones modeled by using ARCore library. Once the alignment between virtual and real worlds is done, the application enables the visualization, navigation and interaction with the virtual facility networks in real-time. Thus, our method may be used by private companies and public institutions responsible of the indoor facilities management and also may be integrated with other applications focused on indoor navigation.

show abstract

“…While this approach is precise and fast, it has several disadvantages, such as the fact that these markers affect user experience and that the tracking is limited to areas where they are visible. This led to the development of markerless tracking [3], which is often based on visual Simultaneous Localization and Mapping (SLAM) [4]. SLAM is the task of constructing and updating a 3D map of an unknown environment, while simultaneously tracking the agent's pose within it.…”

Section: Introductionmentioning

confidence: 99%

EnvSLAM: Combining SLAM Systems and Neural Networks to Improve the Environment Fusion in AR Applications

Marchesi

Eichhorn

Plecher

et al. 2021

IJGI

View full text Add to dashboard Cite

Augmented Reality (AR) has increasingly benefited from the use of Simultaneous Localization and Mapping (SLAM) systems. This technology has enabled developers to create AR markerless applications, but lack semantic understanding of their environment. The inclusion of this information would empower AR applications to better react to the surroundings more realistically. To gain semantic knowledge, in recent years, focus has shifted toward fusing SLAM systems with neural networks, giving birth to the field of Semantic SLAM. Building on existing research, this paper aimed to create a SLAM system that generates a 3D map using ORB-SLAM2 and enriches it with semantic knowledge originated from the Fast-SCNN network. The key novelty of our approach is a new method for improving the predictions of neural networks, employed to balance the loss of accuracy introduced by efficient real-time models. Exploiting sensor information provided by a smartphone, GPS coordinates are utilized to query the OpenStreetMap database. The returned information is used to understand which classes are currently absent in the environment, so that they can be removed from the network’s prediction with the goal of improving its accuracy. We achieved 87.40% Pixel Accuracy with Fast-SCNN on our custom version of COCO-Stuff and showed an improvement by involving GPS data for our self-made smartphone dataset resulting in 90.24% Pixel Accuracy. Having in mind the use on smartphones, the implementation aimed to find a trade-off between accuracy and efficiency, making the system achieve an unprecedented speed. To this end, the system was carefully designed and a strong focus on lightweight neural networks is also fundamental. This enabled the creation of an above real-time Semantic SLAM system that we called EnvSLAM (Environment SLAM). Our extensive evaluation reveals the efficiency of the system features and the operability in above real-time (48.1 frames per second with an input image resolution of 640 × 360 pixels). Moreover, the GPS integration indicates an effective improvement of the network’s prediction accuracy.

show abstract

From Marker to Markerless in Augmented Reality

Cited by 24 publications

References 43 publications

Application of the Deep CNN-Based Method in Industrial System for Wire Marking Identification

Application of the Deep CNN-Based Method in Industrial System for Wire Marking Identification

GEUINF: Real-Time Visualization of Indoor Facilities Using Mixed Reality

EnvSLAM: Combining SLAM Systems and Neural Networks to Improve the Environment Fusion in AR Applications

Contact Info

Product

Resources

About