A Benchmark and Evaluation of Non-Rigid Structure from Motion

Jensen, Sebastian Hoppe Nesgaard; Doest, Mads Emil Brix; Aanæs, Henrik; Bue, Alessio Del

doi:10.1007/s11263-020-01406-y

Cited by 20 publications

(19 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In reality, facial movement can and will violate this assumption, leading to erroneous pose estimation. We are therefore working on developing methods to decouple rigid motion from "mixed" (rigid + non-rigid) motion of the forehead based on traditional computer vision approaches 36,37 and more recent methods involving deep neural networks. 38 However, it is also important to point out that the use of attached markers is not immune from non-rigid motion of the forehead.…”

Section: Discussionmentioning

confidence: 99%

Marker‐free optical stereo motion tracking for in‐bore MRI and PET‐MRI application

et al. 2020

View full text Add to dashboard Cite

Prospective motion correction is arguably the "silver bullet" solution for magnetic resonance imaging (MRI) studies impacted by motion, applicable to almost any pulse sequence and immune from the spin history artifacts introduced by a moving object. In prospective motion correction, the magnetic field gradients and radio frequency waveforms are adjusted in real time in response to measured head motion so as to maintain the head in a stationary reference frame relative to the scanner. Vital for this approach are accurate and rapidly sampled head pose measurements, which may be obtained optically using cameras. However, most optical methods are limited by the need to attach physical markers to the skin, which leads to decoupling of head and marker motion and reduces the effectiveness of correction. In this work we investigate the feasibility and initial performance of a stereo-optical motion tracking method which does not require any attached markers. Methods: The method relies on detecting distinctive natural features or amplified features (using skin stamps) directly on the forehead in multiple camera views, and then deriving pose estimates via a 3D-2D registration between the skin features and a database of forehead landmarks. To demonstrate the feasibility and potential accuracy of the marker-free method for discrete (step-wise) head motion, we performed out-of-bore and in-bore experiments using robotically and manually controlled phantoms in addition to in-bore testing on human volunteers. We also developed a convenient out-of-bore test bed to benchmark and optimize the motion tracking performance. Results: For out-of-bore phantom tests, the pose estimation accuracy (compared to robotic ground truth) was 0.14 mm and 0.23 degrees for incremental translation and rotation, respectively. For arbitrary motion, the pose accuracy obtained using the smallest forehead feature patch was equivalent to 0.21 AE 0.11 mm positional accuracy in the striatum. For in-bore phantom experiments, the accuracy of rigid-body motion parameters (compared to wireless MR-sensitive markers) was 0.

show abstract

Section: Discussionmentioning

confidence: 99%

Marker‐free optical stereo motion tracking for in‐bore MRI and PET‐MRI application

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Non-rigid structure-from-motion techniques estimate relative depth using only images by exploiting statistical and physical heuristics [21]. Here, we focus on the methods that estimate dense depth maps in dynamic scenes, namely [13]- [17].…”

Section: B Non-rigid Structure-from-motion (Nrsfm)mentioning

confidence: 99%

Depth Map Estimation of Dynamic Scenes Using Prior Depth Information

Noraky,

Sze

2020

Preprint

View full text Add to dashboard Cite

Depth information is useful for many applications. Active depth sensors are appealing because they obtain dense and accurate depth maps. However, due to issues that range from power constraints to multi-sensor interference, these sensors cannot always be continuously used. To overcome this limitation, we propose an algorithm that estimates depth maps using concurrently collected images and a previously measured depth map for dynamic scenes, where both the camera and objects in the scene may be independently moving. To estimate depth in these scenarios, our algorithm models the dynamic scene motion using independent and rigid motions. It then uses the previous depth map to efficiently estimate these rigid motions and obtain a new depth map. Our goal is to balance the acquisition of depth between the active depth sensor and computation, without incurring a large computational cost. Thus, we leverage the prior depth information to avoid computationally expensive operations like dense optical flow estimation or segmentation used in similar approaches. Our approach can obtain dense depth maps at up to real-time (30 FPS) on a standard laptop computer, which is orders of magnitude faster than similar approaches. When evaluated using RGB-D datasets of various dynamic scenes, our approach estimates depth maps with a mean relative error of 2.5% while reducing the active depth sensor usage by over 90%.

show abstract

“…While benchmarking datasets from the general-purpose 3D reconstruction community exist and can be used as video inputs for virtual endoscopy algorithms [28] – [32] , their features do not resemble biological tissue nor do the movements and optical properties of commercial cameras mimic those of an endoscope. Hence, evaluations using these datasets do not generalize well to the clinical domain [32] .…”

Section: Introductionmentioning

confidence: 99%

Cost-Efficient Video Synthesis and Evaluation for Development of Virtual 3D Endoscopy

Zhou

Eimen

Seibel

et al. 2021

IEEE J. Transl. Eng. Health Med.

View full text Add to dashboard Cite

Objective: 3D reconstruction of the shape and texture of hollow organs captured by endoscopy is important for the diagnosis and surveillance of early and recurrent cancers. Better evaluation of 3D reconstruction pipelines developed for such applications requires easy access to extensive datasets and associated ground truths, cost-efficient and scalable simulations of a range of possible clinical scenarios, and more reliable and insightful metrics to assess performance. Methods: We present a computer-aided simulation platform for cost-effective synthesis of monocular endoscope videos and corresponding ground truths that mimic a range of potential settings and situations one might encounter during acquisition of clinical endoscopy videos. Using cystoscopy of the bladder as model case, we generated an extensive dataset comprising several synthesized videos of a bladder phantom. We then introduce a novel evaluation procedure to reliably assess an individual 3D reconstruction pipeline or to compare different pipelines. Results: To illustrate the use of the proposed platform and evaluation procedure, we use the aforementioned dataset and ground truths to evaluate a proprietary 3D reconstruction pipeline (CYSTO3D) for bladder cystoscopy videos and compared it with a general-purpose 3D reconstruction pipeline (COLMAP). The evaluation results provide insight into the suggested clinical acquisition protocol and several potential areas for refinement of the pipeline to improve future performance. Conclusion: Our work proposes an endoscope video synthesis and reconstruction evaluation toolset and presents experimental results that illustrate usage of the toolset to efficiently assess performance and reveal possible problems of any given 3D reconstruction pipeline, to compare different pipelines, and to provide technically or clinically actionable insights.

show abstract

A Benchmark and Evaluation of Non-Rigid Structure from Motion

Cited by 20 publications

References 59 publications

Marker‐free optical stereo motion tracking for in‐bore MRI and PET‐MRI application

Marker‐free optical stereo motion tracking for in‐bore MRI and PET‐MRI application

Depth Map Estimation of Dynamic Scenes Using Prior Depth Information

Cost-Efficient Video Synthesis and Evaluation for Development of Virtual 3D Endoscopy

Contact Info

Product

Resources

About