Background including a long-period fast illumination variation is commonly assumed to be foreground by mistake. To solve this problem, proposed is a semantic-based hierarchical Gaussian mixture model integrated with an illumination detection approach. First, autocorrelation-based features for broad identification of background lighting changes and foreground in short-term sequences are presented. Then, the hierarchical Gaussians representing different background illumination variations are maintained. The effectiveness of the proposed method is demonstrated using experiments on pedestrian detection in fast lighting change.Introduction: Background subtraction is a commonly used approach to foreground detection. However, fast illumination variation of background presents a challenge for accurate detection results of foreground. Ordinarily, fast lighting change is considered to appear in regions rather than at pixels. Therefore, many approaches focus on the integration of regional information into a pixel-wise background model. The Gaussian mixture model (GMM) [1] is the most representative background model incorporated with other methods [2-4], so we are mainly concerned with the GMM in this Letter. Tian et al. brought a regional texture measure into the GMM (TGMM) and claimed the solution to accurate foreground detection in fast illumination variation [2]. A hierarchical GMM (HGMM) of different scales was devised to discriminate fast illumination change from foreground [3]. Meanwhile, a memorising GMM (MGMM) was proposed to tackle fast lighting change at pixels in a scene [4]. In the MGMM, Gaussians representing background were labelled and recorded in long-term memory. Actually, fast lighting change results from temporal intensity variations at pixels rather than those in regions. Thus, it is advisable to memorise the former background states deriving from fast lighting change. However, it is unadvisable to label the similar background states excessively without any intended discrimination between different illumination variations.In this Letter, we propose a semantic-based hierarchical GMM (S-HGMM) for long-term memory of periodical fast lighting change. A criterion for illumination detection is first made to identify different lighting changes in short-term sequences. Then, we design a hierarchical GMM including a sequence of short-term Gaussians and a single long-term one tagged semantically with a label of fast lighting change according to the illumination detection results.Background: The GMM is a pixel-wise mixture of K Gaussian distributions representing the recent states. The probability of intensity X t is P(X t ) = K k=1 v k,t × h(X t , m k,t , S k,t ), where h obeys the Gaussian distribution. m k,t , S k,t and v k,t represent the current mean, the covariance matrix and the weight estimation of Gaussian k, respectively. The covariance matrix S k,t is referred to as S k,t = s k,t × I, assuming the red, green and blue pixel values to be independent. A new pixel value X t + 1 is checked against each Gaussi...