A Multimodal Perception-Driven Self Evolving Autonomous Ground Vehicle

Roche, Jamie; De-Silva, Varuna; Kondoz, A.M.

doi:10.1109/tcyb.2021.3113804

Cited by 11 publications

(5 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The LboroAV2 multimodal dataset is collected specifically for this project. An AGV [ 36 ] is used to collect the dataset. The AGV was autonomously driven throughout the data collection period, with minimal human interaction, on structured and unstructured roads on the privately owned Here East compound in Queen Elizabeth Olympic Park, London.…”

Section: Methodsmentioning

confidence: 99%

Towards Interpretable Camera and LiDAR Data Fusion for Autonomous Ground Vehicles Localisation

Tibebu

Silva

Artaud

et al. 2022

Sensors

View full text Add to dashboard Cite

Recent deep learning frameworks draw strong research interest in application of ego-motion estimation as they demonstrate a superior result compared to geometric approaches. However, due to the lack of multimodal datasets, most of these studies primarily focused on single-sensor-based estimation. To overcome this challenge, we collect a unique multimodal dataset named LboroAV2 using multiple sensors, including camera, light detecting and ranging (LiDAR), ultrasound, e-compass and rotary encoder. We also propose an end-to-end deep learning architecture for fusion of RGB images and LiDAR laser scan data for odometry application. The proposed method contains a convolutional encoder, a compressed representation and a recurrent neural network. Besides feature extraction and outlier rejection, the convolutional encoder produces a compressed representation, which is used to visualise the network’s learning process and to pass useful sequential information. The recurrent neural network uses this compressed sequential data to learn the relationship between consecutive time steps. We use the Loughborough autonomous vehicle (LboroAV2) and the Karlsruhe Institute of Technology and Toyota Institute (KITTI) Visual Odometry (VO) datasets to experiment and evaluate our results. In addition to visualising the network’s learning process, our approach provides superior results compared to other similar methods. The code for the proposed architecture is released in GitHub and accessible publicly.

show abstract

Section: Methodsmentioning

confidence: 99%

Towards Interpretable Camera and LiDAR Data Fusion for Autonomous Ground Vehicles Localisation

Tibebu

Silva

Artaud

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…The aspect of path optimization becomes crucial in this case, as it allows for more efficient use of the available infrastructure. This is possible by appropriately using software adapted to the place of application based on VLC (Visible Light Communication) [134], a heuristic model [128], color Petri nets [136], and FSD (Free Space Detection) algorithms [135].…”

Section: Designing a Safe Work Environmentmentioning

confidence: 99%

Interaction between a Human and an AGV System in a Shared Workspace—A Literature Review Identifying Research Areas

Tubis,

Poturaj,

Smok

2024

Sustainability

View full text Add to dashboard Cite

Background: This article presents the results of a literature review from 2018 to 2023, which focused on research related to human and AGV system cooperation in a shared workspace. This study defines AGV systems as systems using Automated Guided Vehicles or Autonomous Guided Vehicles. An Automated Guided Vehicle is a cart that follows a guided path, while an Autonomous Guided Vehicle is an Automated Guided Vehicle that is autonomously controlled. The analyses conducted answered two research questions: (RQ1) In what aspects are the human factor examined in publications on the implementation and operation of AGV systems? (RQ2) Has the human-AGV collaboration aspect been analyzed in the context of a sustainable work environment? Methods: The literature review was conducted following the systematic literature review method, using the PRISMA approach. Results: Based on the search of two journal databases, according to the indicated keywords, 1219 documents pertaining to the analyzed issues were identified. The selection and elimination of documents that did not meet the defined criteria made it possible to limit the number of publications to 117 articles and proceedings papers. On this basis, the authors defined a classification framework comprising five basic research categories and nine subcategories. The analyzed documents were classified, and each distinguished group was characterized by describing the results. Conclusions: The development of a two-level classification framework for research from the analyzed area according to the assumptions of the concept map and the identification of research gaps in the area of human-AGV interaction.

show abstract

“…Through the years, interest in combining various sensors to achieve higher accuracy and efficiency has been widespread. Many studies regarding sensor fusions have been successfully integrated and applied in multiple fields, such as camera-lidar integration for semantic mapping [ 47 ], driver aid systems for intelligent vehicles [ 48 , 49 ], target tracking for robotic fish [ 50 ], activity detection of sound sources [ 51 ] and avian monitoring [ 52 ]. An underwater acoustic-optic image matching was proposed by Zhou et al [ 53 ].…”

Section: Related Workmentioning

confidence: 99%

A Two-Mode Underwater Smart Sensor Object for Precision Aquaculture Based on AIoT Technology

Chang

Ubina

Cheng

et al. 2022

Sensors

View full text Add to dashboard Cite

Monitoring the status of culture fish is an essential task for precision aquaculture using a smart underwater imaging device as a non-intrusive way of sensing to monitor freely swimming fish even in turbid or low-ambient-light waters. This paper developed a two-mode underwater surveillance camera system consisting of a sonar imaging device and a stereo camera. The sonar imaging device has two cloud-based Artificial Intelligence (AI) functions that estimate the quantity and the distribution of the length and weight of fish in a crowded fish school. Because sonar images can be noisy and fish instances of an overcrowded fish school are often overlapped, machine learning technologies, such as Mask R-CNN, Gaussian mixture models, convolutional neural networks, and semantic segmentation networks were employed to address the difficulty in the analysis of fish in sonar images. Furthermore, the sonar and stereo RGB images were aligned in the 3D space, offering an additional AI function for fish annotation based on RGB images. The proposed two-mode surveillance camera was tested to collect data from aquaculture tanks and off-shore net cages using a cloud-based AIoT system. The accuracy of the proposed AI functions based on human-annotated fish metric data sets were tested to verify the feasibility and suitability of the smart camera for the estimation of remote underwater fish metrics.

show abstract

A Multimodal Perception-Driven Self Evolving Autonomous Ground Vehicle

Cited by 11 publications

References 43 publications

Towards Interpretable Camera and LiDAR Data Fusion for Autonomous Ground Vehicles Localisation

Towards Interpretable Camera and LiDAR Data Fusion for Autonomous Ground Vehicles Localisation

Interaction between a Human and an AGV System in a Shared Workspace—A Literature Review Identifying Research Areas

A Two-Mode Underwater Smart Sensor Object for Precision Aquaculture Based on AIoT Technology

Contact Info

Product

Resources

About