In this paper, we propose an algorithm that automatically detects and removes moving logos in video. The proposed algorithm consists of logo detection, tracking and extraction, and removal. In the logo detection step, we automatically detect moving logos using the saliency map and the scale invariant feature transform. In the logo tracking and extraction step, an improved mean shift tracking method and backward tracking technique are used in which only logo regions are extracted by using color information. Finally, in the logo removal step, the detected logo region is filled with the region of neighborhood using an exemplar-based inpainting that can fill a large detected region without artifacts. These steps are effectively interconnected using control flags. Experimental results with various test sequences show that the proposed algorithm is effective to detect, track, and remove moving logos in video.