6D-ViCuT: Six degree-of-freedom visual cuboid tracking dataset for manual packing of cargo in warehouses

Camacho-Muñoz, Guillermo A.; Franco, Jose; Nope-Rodríguez, Sandra Esperanza; Correa, Humberto Loaiza; Parga, Sebastian Gil; Álvarez‐Martínez, David

doi:10.1016/j.dib.2023.109385

Cited by 3 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The camera speed also varies at two levels: low speed (S low ) and high speed (S high ). The evaluations are conducted on the dataset provided by [35], and the metrics of interest are the precision, recall, and F1-score of the 6D plane segment tracking problem. The parameters used to configure the algorithms are presented in Table 5.…”

Section: Resultsmentioning

confidence: 99%

“…Packing involves the action of picking a box from the consolidation zone and transferring it to the packing zone (as depicted in Figure 8), repeating this process until no more boxes remain in the consolidation zone. Sample images from the dataset can be consulted in [35].…”

Section: Datasetmentioning

confidence: 99%

“…Table 3 presents descriptors of the 18 assessed sessions. The "session ID" column contains the identifier of each session in the original dataset [35], and the next column contains the number of frames comprising each session. The "camera speed" column includes descriptors of the mean speed, standard deviation, and maximum speed reached during the session.…”

Section: Datasetmentioning

confidence: 99%

“…The following column describes the number of boxes (Q) at the beginning of each session, aiming to have variations in texture, size (heterogeneity), and deformation state of the boxes. The last column contains the order in which the boxes were extracted from the consolidation area relative to the identifier Although the dataset contains images recorded by three capture systems [35], this work only processes images from two of these systems: (1) the HoloLens 2 system located on the operator's head, which provides the pairs {p(k), P c h (k)}; the evaluated algorithms process colorless point clouds, therefore, the original format of the point cloud (which includes color information) was converted to a colorless format. (2) The MoCap system, which allows obtaining the poses P m i (k) of the planes belonging to boxes located in the consolidation area, which are relevant for the application.…”

Section: Datasetmentioning

confidence: 99%

“…This set of poses is used as ground truth while evaluating the 6D tracking system performance. For additional details about the devices and image capture methods, please refer to [35]. The dataset is complemented with annotations of planes visible from the Holo Lens 2 camera in each frame.…”

Section: Datasetmentioning

confidence: 99%

See 4 more Smart Citations

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Muñoz,

Nope-Rodríguez,

Loaiza-Correa

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Available solutions to assist human operators in cargo packing processes offer alternatives to maximize the spatial occupancy of containers used in intralogistics. However, these solutions consist of sequential instructions for picking each box and positioning it in the containers, making it challenging for an operator to interpret and requiring them to alternate between reading the instructions and executing the task. A potential solution to these issues lies in a tool that naturally communicates each box's initial and final location in the desired sequence to the operator. While 6D visual object tracking systems have demonstrated good performance, they have yet to be evaluated in real-world scenarios of manual box packing. They also need to use the available prior knowledge of the packing operation, such as the number of boxes, box size, and physical packing sequence. This study explores the inclusion of box size priors in 6D plane segment tracking systems driven by images from moving cameras and quantifies their contribution in terms of tracker performance when assessed in manual box packing operations. To do this, it compares the performance of a plane segment tracking system, considering variations in the tracking algorithm and camera speed (onboard the packing operator) during the mapping of a manual cargo packing process. The tracking algorithm varies at two levels: algorithm (Awpk), which integrates prior knowledge of box sizes in the scene, and algorithm (Awoutpk), which assumes ignorance of box properties. Camera speed is also evaluated at two levels: low speed (Slow) and high speed (Shigh). This study analyzes the impact of these factors on the precision, recall, and F1-score of the plane segment tracking system. ANOVA analysis was applied to the precision and F1-score results, which allows determining that neither the camera speed-algorithm interactions nor the camera speed are significant in the precision of the tracking system. The factor that presented a significant effect is the tracking algorithm. Tukey's pairwise comparisons concluded that the precision and F1-score of each algorithm level are significantly different, with algorithm Awpk being superior in each evaluation. This superiority reaches its maximum in the tracking of top plane segments: 22 and 14 percentage units for precision and F1-score metrics, respectively. However, the results on the recall metric remain similar with and without the addition of prior knowledge. The contribution of including prior knowledge of box sizes in (6D) plane segment tracking algorithms is identified in reducing false positives. This reduction is associated with significant increases in the tracking system's precision and F1-score metrics. Future work will investigate whether the identified benefits propagate to the tracking problem on objects composed of plane segments, such as cubes or boxes.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

See 3 more Smart Citations

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Muñoz,

Nope-Rodríguez,

Loaiza-Correa

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Camacho-Muñoz,

Nope Rodríguez,

Loaiza-Correa

et al. 2024

J Image Video Proc.

View full text Add to dashboard Cite

This paper addresses the problem of 6D pose tracking of plane segments from point clouds acquired from a mobile camera. This is motivated by manual packing operations, where an opportunity exists to enhance performance, aiding operators with instructions based on augmented reality. The approach uses as input point clouds, by its advantages for extracting geometric information relevant to estimating the 6D pose of rigid objects. The proposed algorithm begins with a RANSAC fitting stage on the raw point cloud. It then implements strategies to compute the 2D size and 6D pose of plane segments from geometric analysis of the fitted point cloud. Redundant detections are combined using a new quality factor that predicts point cloud mapping density and allows the selection of the most accurate detection. The algorithm is designed for dynamic scenes, employing a novel particle concept in the point cloud space to track detections' validity over time. A variant of the algorithm uses box size priors (available in most packing operations) to filter out irrelevant detections. The impact of this prior knowledge is evaluated through an experimental design that compares the performance of a plane segment tracking system, considering variations in the tracking algorithm and camera speed (onboard the packing operator). The tracking algorithm varies at two levels: algorithm ( A wpk ), which integrates prior knowledge of box sizes, and algorithm ( A woutpk ), which assumes ignorance of box properties. Camera speed is evaluated at low and high speeds. Results indicate increments in the precision and F1-score associated with using the A wpk algorithm and consistent performance across both velocities. These results confirm the enhancement of the performance of a tracking system in a real-life and complex scenario by including previous knowledge of the elements in the scene. The proposed algorithm is limited to tracking plane segments of boxes fully supported on surfaces parallel to the ground plane and not stacked. Future works are proposed to include strategies to resolve this limitation.

show abstract

Generative artificial intelligence of things systems, multisensory immersive extended reality technologies, and algorithmic big data simulation and modelling tools in digital twin industrial metaverse

Kliestik,

Kral,

Bugaj

et al. 2024

View full text Add to dashboard Cite

Research background: Multi-modal synthetic data fusion and analysis, simulation and modelling technologies, and virtual environmental and location sensors shape the industrial metaverse. Visual digital twins, smart manufacturing and sensory data mining techniques, 3D digital twin simulation modelling and predictive maintenance tools, big data and mobile location analytics, and cloud-connected and spatial computing devices further immersive virtual spaces, decentralized 3D digital worlds, synthetic reality spaces, and the industrial metaverse. Purpose of the article: We aim to show that big data computing and extended cognitive systems, 3D computer vision-based production and cognitive neuro-engineering technologies, and synthetic data interoperability improve artificial intelligence-based digital twin industrial metaverse and hyper-immersive simulated environments. Geolocation data mining and tracking tools, image processing computational and robot motion algorithms, and digital twin and virtual immersive technologies shape the economic and business management of extended reality environments and the industrial metaverse. Methods: Quality tools: AMSTAR, BIBOT, CASP, Catchii, R package and Shiny app citationchaser, DistillerSR, JBI SUMARI, Litstream, Nested Knowledge, Rayyan, and Systematic Review Accelerator. Search period: April 2024. Search terms: “digital twin industrial metaverse” + “artificial Intelligence of Things systems”, “multisensory immersive extended reality technologies”, and “algorithmic big data simulation and modelling tools”. Selected sources: 114 out of 336. Published research inspected: 2022–2024. PRISMA was the reporting quality assessment tool. Dimensions and VOSviewer were deployed as data visualization tools. Findings & value added: Simulated augmented reality and multi-sensory tracking technologies, explainable artificial intelligence-based decision support and cloud-based robotic cooperation systems, and ambient intelligence and deep learning-based predictive analytics modelling tools are instrumental in augmented reality environments and in the industrial metaverse. The economic and business management of the industrial metaverse necessitates connected enterprise production and big data computing systems, simulation and modelling technologies, and virtual reality-embedded digital twins.

show abstract

6D-ViCuT: Six degree-of-freedom visual cuboid tracking dataset for manual packing of cargo in warehouses

Cited by 3 publications

References 5 publications

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Evaluation of the use of box size priors for 6D plane segment tracking from point clouds with applications in cargo packing

Generative artificial intelligence of things systems, multisensory immersive extended reality technologies, and algorithmic big data simulation and modelling tools in digital twin industrial metaverse

Contact Info

Product

Resources

About