The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

Morales, José; Vázquez-Martín, Ricardo; Mandow, Anthony; Morilla-Cabello, David; García-Cerezo, Alfonso

doi:10.1177/02783649211004959

Cited by 19 publications

(16 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…En un trabajo previo hemos comprobado la viabilidad de utilizar un re-entrenamiento de imágenes TIR sobre una red YOLO [5]. Resulta interesante estudiar el rendimiento de la red RGB entrenada en este trabajo para detectar objetos en imágenes TIR del dataset UMA-SAR [14]. Los resultados obtenidos se ofrecen en la Tabla 11, donde no se ha incluido la clase víctima ya que su presencia no es representativa en en las imágenes de prueba.…”

Section: Resultsunclassified

See 1 more Smart Citation

Análisis de técnicas de aumento de datos y entrenamiento en YOLOv3 para detección de objetos en imágenes RGB y TIR del UMA-SAR Dataset

Vázquez-Martín¹,

Mandow²,

Morales-Rodriguez³

et al. 2021

Xlii Jornadas De Automática : Libro De Actas

Self Cite

View full text Add to dashboard Cite

El uso de imágenes de los espectros visible (RGB) e infrarrojo térmico (TIR) para la detección de objetos puede resultar crucial en aplicaciones donde las condiciones de visibilidad están limitadas, como la robótica para búsqueda y rescate en catástrofes. Para ello resulta beneficioso analizar cómo las técnicas de aprendizaje profundo basadas en redes neuronales convolucionales (CNN) pueden aplicarse a ambas modalidades. En este artículo se analizan diferentes configuraciones y parámetros para el entrenamiento de CNN tanto para imágenes térmicas como para imágenes equivalentes del espectro visible. En concreto, se aborda el problema del sobre-entrenamiento para determinar una configuración eficaz de técnicas de aumento de datos y parada temprana. El caso de estudio se ha realizado con la red de código abierto YOLOv3, pre-entrenada con el dataset RGB COCO y optimizada (o re-entrenada) con el conjunto público de datos UMA-SAR dataset, que incluye pares de imágenes RGB y TIR obtenidas en ejercicios realistas de rescate.

show abstract

Section: Resultsunclassified

“…Para el entrenamiento hemos utilizado nuestro conjunto de datos UMA-SAR Dataset [14], disponible públicamente en www.uma.es/robotics-andmechatronics/sar-datasets.…”

Section: El Dataset Uma-sarunclassified

Análisis de técnicas de aumento de datos y entrenamiento en YOLOv3 para detección de objetos en imágenes RGB y TIR del UMA-SAR Dataset

Vázquez-Martín¹,

Mandow²,

Morales-Rodriguez³

et al. 2021

Xlii Jornadas De Automática : Libro De Actas

Self Cite

View full text Add to dashboard Cite

show abstract

“…Over 150 people from these services participated in the edition of 2021. These annual exercises provide an effective framework for the testing and assessing new technologies with the cooperation of actual responders [ 41 , 64 ].…”

Section: Methodsmentioning

confidence: 99%

“…In addition to smartphones, other equipment on board robots, such as different kinds of cameras, LiDARs or GPS, could be used to communicate other sensory information via ROS [ 41 ]. Conversely, the X-IoCA architecture makes it possible to transmit this heavy information through the WAN, in real time, thanks to the 5G technology characteristics [ 42 , 43 , 44 ] that support it.…”

Section: Sensory and Communication System Architecture: X-iocamentioning

confidence: 99%

The Internet of Cooperative Agents Architecture (X-IoCA) for Robots, Hybrid Sensor Networks, and MEC Centers in Complex Environments: A Search and Rescue Case Study

Bravo-Arrabal

Toscano-Moreno

Lozano

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

Cloud robotics and advanced communications can foster a step-change in cooperative robots and hybrid wireless sensor networks (H-WSN) for demanding environments (e.g., disaster response, mining, demolition, and nuclear sites) by enabling the timely sharing of data and computational resources between robot and human teams. However, the operational complexity of such multi-agent systems requires defining effective architectures, coping with implementation details, and testing in realistic deployments. This article proposes X-IoCA, an Internet of robotic things (IoRT) and communication architecture consisting of a hybrid and heterogeneous network of wireless transceivers (H2WTN), based on LoRa and BLE technologies, and a robot operating system (ROS) network. The IoRT is connected to a feedback information system (FIS) distributed among multi-access edge computing (MEC) centers. Furthermore, we present SAR-IoCA, an implementation of the architecture for search and rescue (SAR) integrated into a 5G network. The FIS for this application consists of an SAR-FIS (including a path planner for UGVs considering risks detected by a LoRa H-WSN) and an ROS-FIS (for real-time monitoring and processing of information published throughout the ROS network). Moreover, we discuss lessons learned from using SAR-IoCA in a realistic exercise where three UGVs, a UAV, and responders collaborated to rescue victims from a tunnel accessible through rough terrain.

show abstract

“…Compared with video and image action recognition, skeletal point data is more robust to lighting conditions, viewpoint transformations, and changes in human form. Moreover, in complex pure action recognition (action recognition without the influence of other objects), skeleton point data has an advantage that video data cannot match ( Morales et al, 2021 ). Regarding the action recognition of skeletal points, the traditional method of manually designed features usually represents the motion of several human skeletal nodes as the action of the whole human body.…”

Section: Current Status Of Researchmentioning

confidence: 99%

Multimodal Art Pose Recognition and Interaction With Human Intelligence Enhancement

Liu

Dang

2021

Front. Psychol.

View full text Add to dashboard Cite

This paper provides an in-depth study and analysis of human artistic poses through intelligently enhanced multimodal artistic pose recognition. A complementary network model architecture of multimodal information based on motion energy proposed. The network exploits both the rich information of appearance features provided by RGB data and the depth information provided by depth data as well as the characteristics of robustness to luminance and observation angle. The multimodal fusion is accomplished by the complementary information characteristics of the two modalities. Moreover, to better model the long-range temporal structure while considering action classes with sub-action sharing phenomena, an energy-guided video segmentation method is employed. And in the feature fusion stage, a cross-modal cross-fusion approach is proposed, which enables the convolutional network to share local features of two modalities not only in the shallow layer but also to obtain the fusion of global features in the deep convolutional layer by connecting the feature maps of multiple convolutional layers. Firstly, the Kinect camera is used to acquire the color image data of the human body, the depth image data, and the 3D coordinate data of the skeletal points using the Open pose open-source framework. Then, the action automatically extracted from keyframes based on the distance between the hand and the head, and the relative distance features are extracted from the keyframes to describe the action, the local occupancy pattern features and HSV color space features are extracted to describe the object, and finally, the feature fusion is performed and the complex action recognition task is completed. To solve the consistency problem of virtual-reality fusion, the mapping relationship between hand joint point coordinates and the virtual scene is determined in the augmented reality scene, and the coordinate consistency model of natural hand and virtual model is established; finally, the real-time interaction between hand gesture and virtual model is realized, and the average correct rate of its hand gesture reaches 99.04%, which improves the robustness and real-time interaction of hand gesture recognition.

show abstract

The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

Cited by 19 publications

References 37 publications

Análisis de técnicas de aumento de datos y entrenamiento en YOLOv3 para detección de objetos en imágenes RGB y TIR del UMA-SAR Dataset

Análisis de técnicas de aumento de datos y entrenamiento en YOLOv3 para detección de objetos en imágenes RGB y TIR del UMA-SAR Dataset

The Internet of Cooperative Agents Architecture (X-IoCA) for Robots, Hybrid Sensor Networks, and MEC Centers in Complex Environments: A Search and Rescue Case Study

Multimodal Art Pose Recognition and Interaction With Human Intelligence Enhancement

Contact Info

Product

Resources

About