Robustness is an important factor for background modelling on various scenarios. Current pixel-based adaptive segmentation method cannot effectively tackle diverse objects simultaneously. To address this problem, in this study, a background modelling method using discriminative motion representation is proposed. Instead of simple usage of intensity to construct the background model, the proposed method extracts a new local descriptor which uses a weighted combination of differential excitations for each pixel to enhance the discriminability of pixels. On the basis of this background model, different categories of objects can be quickly identified by a simple but effective classification rule and accurately be represented in background model by a smart selection of updating strategies. Therefore, the authors' background modelling method can generate complete representation for static objects and decrease false detection caused by dynamic background or illumination variations. Extensive experiments have been conducted to demonstrate that the proposed method obtains more advantages of foreground detection than the state-of-the-art methods. In addition, the proposed method provides a computational efficient algorithm for foreground detection tasks.