Feature extraction of electrocardiogram (ECG) is the fundamental work of further automatic diagnosis. However, suffered from various kinds of noises with white, pink, and other colors, feature extraction is not a straightforward work but requires necessary signal processing techniques. In this paper, we propose an accurate and robust ECG feature extraction method based on mean shift algorithm which has the ability to remove noise involved in input signal by taking advantage of its embedded Gaussian filter and locate extremes of input signal using gradient optimization based on self-adaptive search steps. To demonstrate the availability and efficacy of the proposed method, we conduct experiments on signals contaminated by noises of white, pink and brown colors from 5dB to 15dB signal-noise ratios. Clean signals are produced by ECG synthesizer (ECGSyn) so that we can obtain the real features and quantitatively calculate feature extraction errors of the proposed method. Experiment results verify that our method can handle various kinds of noises and achieve satisfactory feature extraction performance.