3D thermal mapping of building interiors using an RGB-D and thermal camera

Vidas, Stephen; Moghadam, Peyman; Bosse, Michael

doi:10.1109/icra.2013.6630890

Cited by 127 publications

(80 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Parameters were found by simply viewing an overlay of the two image streams, visual and thermal, and pressing keys to alter parameters until images were aligned. Although more robust and complex approaches could be followed, such as compensating for intrinsic parameters such as lens distortion and calibrating with a thermal mask held in front of a heat source (Vidas et al, 2013), we expected our approach would be sufficient to consummate our goal of obtaining some basic insight regarding the feasibility of thermovisual inference of medicine intake. …”

Section: Detecting Touches To Objectsmentioning

confidence: 99%

PastVision+: Thermovisual Inference of Recent Medicine Intake by Detecting Heated Objects and Cooled Lips

Cooney

Bigün

2017

Front. Robot. AI

View full text Add to dashboard Cite

This article addresses the problem of how a robot can infer what a person has done recently, with a focus on checking oral medicine intake in dementia patients. We present PastVision+, an approach showing how thermovisual cues in objects and humans can be leveraged to infer recent unobserved human-object interactions. Our expectation is that this approach can provide enhanced speed and robustness compared to existing methods, because our approach can draw inferences from single images without needing to wait to observe ongoing actions and can deal with short-lasting occlusions; when combined, we expect a potential improvement in accuracy due to the extra information from knowing what a person has recently done. To evaluate our approach, we obtained some data in which an experimenter touched medicine packages and a glass of water to simulate intake of oral medicine, for a challenging scenario in which some touches were conducted in front of a warm background. Results were promising, with a detection accuracy of touched objects of 50% at the 15 s mark and 0% at the 60 s mark, and a detection accuracy of cooled lips of about 100 and 60% at the 15 s mark for cold and tepid water, respectively. Furthermore, we conducted a follow-up check for another challenging scenario in which some participants pretended to take medicine or otherwise touched a medicine package: accuracies of inferring object touches, mouth touches, and actions were 72.2, 80.3, and 58.3% initially, and 50.0, 81.7, and 50.0% at the 15 s mark, with a rate of 89.0% for person identification. The results suggested some areas in which further improvements would be possible, toward facilitating robot inference of human actions, in the context of medicine intake monitoring.

show abstract

Section: Detecting Touches To Objectsmentioning

confidence: 99%

PastVision+: Thermovisual Inference of Recent Medicine Intake by Detecting Heated Objects and Cooled Lips

Cooney

Bigün

2017

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…They also generate RGB point clouds on which the thermal information is added. A system using and RGB-D camera and a single additional thermal camera is presented in [24], where after computing the poses corresponding to each RGB and IRT image, raycasting is used to map RGB and IRT intensities to the voxels reconstructed from the range sensor. Even though depth cameras and range scanners can provide accurate point cloud with low processing costs, their precision and range can be limited when operated outdoors depending on the technology employed [1,13].…”

Section: Related Workmentioning

confidence: 99%

Registration of RGB and Thermal Point Clouds Generated by Structure From Motion

Truong

Yamawaki

Mori

et al. 2017

2017 IEEE International Conference on Computer Vision Workshops (ICCVW)

View full text Add to dashboard Cite

Thermal imaging has become a valuable tool in various fields for remote sensing and can provide relevant information to perform object recognition or classification. In this paper, we present an automated method to obtain a 3D model fusing data from a visible and a thermal camera. The RGB and thermal point clouds are generated independently by structure from motion. The registration process includes a normalization of the point cloud scale, a global registration based on calibration data and the output of the structure from motion, and a fine registration employing a variant of the Iterative Closest Point optimization. Experimental results demonstrate the accuracy and robustness of the overall process.

show abstract

“…Vidas et al [3], as well as Matsumoto et al [4] mount a Kinect with a thermalinfrared camera on a hand-held stereo rig. They obtain a 3D structure of the scene using KinectFusion [5].…”

Section: Related Workmentioning

confidence: 99%

Superimposing Thermal-Infrared Data on 3D Structure Reconstructed by RGB Visual Odometry

Yamawaki

Truong

Mori

et al. 2018

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYIn this paper, we propose a method to generate a threedimensional (3D) thermal map and RGB + thermal (RGB-T) images of a scene from thermal-infrared and RGB images. The scene images are acquired by moving both a RGB camera and an thermal-infrared camera mounted on a stereo rig. Before capturing the scene with those cameras, we estimate their respective intrinsic parameters and their relative pose. Then, we reconstruct the 3D structures of the scene by using Direct Sparse Odometry (DSO) using the RGB images. In order to superimpose thermal information onto each point generated from DSO, we propose a method for estimating the scale of the point cloud corresponding to the extrinsic parameters between both cameras by matching depth images recovered from the RGB camera and the thermal-infrared camera based on mutual information. We also generate RGB-T images using the 3D structure of the scene and Delaunay triangulation. We do not rely on depth cameras and, therefore, our technique is not limited to scenes within the measurement range of the depth cameras. To demonstrate this technique, we generate 3D thermal maps and RGB-T images for both indoor and outdoor scenes.

show abstract

3D thermal mapping of building interiors using an RGB-D and thermal camera

Cited by 127 publications

References 11 publications

PastVision+: Thermovisual Inference of Recent Medicine Intake by Detecting Heated Objects and Cooled Lips

PastVision+: Thermovisual Inference of Recent Medicine Intake by Detecting Heated Objects and Cooled Lips

Registration of RGB and Thermal Point Clouds Generated by Structure From Motion

Superimposing Thermal-Infrared Data on 3D Structure Reconstructed by RGB Visual Odometry

Contact Info

Product

Resources

About