In the paper, a new approach to the impulsive noise removal in color images is presented. The new filtering design is based on the peer group concept, which determines the membership of a central pixel of the filtering window to its local neighborhood, in terms of the number of close pixels. Two pixels are declared as close if their distance in a given color space does not exceed a predefined threshold value. A pixel is treated as not corrupted by the impulsive noise process, if its peer group consists of at least two close pixels, otherwise this pixel is replaced by a weighted average of uncorrupted samples from the local neighborhood. The peer group size assigned to each pixel is used for the averaging operation, so that pixels which have many peers are taken with higher weight. The new filtering design proved to restore efficiently color images corrupted by even strong impulsive noise, while preserving tiny image details. The beneficial property of the proposed filter is its very low computational complexity, which allows its application in real-time image processing tasks.