Toward efficient background reconstruction for 3D-view synthesis in dynamic scenes

Inpainting surgical occlusion from laparoscopic video sequences for robot-assisted interventions

“…The first method uses video object removal algorithms 29 , 30 to replace the segmented tool pixels in the current frame with data from previous frames, which we have mentioned in our prior work 31 . The procedure establishes dense correspondences (optical flow) between the pixels (i.e., regions) observed in the background region of a previous frame It(x,y) and the pixels (i.e., regions) occluded by the surgical tool in the present frame It−1(x,y).…”

“…To avoid this problem, we define a cumulative mapping function boldVt(x), 29 , 30 which, for each pixel, stores the index of the source frame I1,I2,…,It−1 and the spatial shift relative to the source background region where the pixel was last visible. This mitigates the blurriness problem because source pixels used to inpaint the tool region are now being copied once via interpolation as opposed to multiple times.…”