With more and more cameras installed in public places, video surveillance systems play an increasingly important role in public safety. Research on intelligent video monitoring, especially activity recognition, is attracting increasing attention in the field of image processing. Unlike activity recognition of a single tracking object, group activity is more complex and difficult to recognize. To design a fast realtime group activity recognition algorithm without other auxiliary data, low computational cost is our focus. There are four steps for our group activity recognition system: preprocessing the captured videos, extracting foregrounds from backgrounds, tracking multiple objects and recognizing group activity. To remove noise in each frame image, the combination of the Gaussian filter algorithm and median filter algorithm is used in the preprocessing step. Then, the Gaussian mixture model is adopted to extract the foreground image. To ensure low computational cost, real-time Cam-Shift is chosen to track group activity with morphological operations in the tracking step. In the recognition step, the changing histogram rate is defined as the measure of identifying group behavior. Here, the changing histogram rate refers to the number of changing histograms and changing proportions. Experimental results show that the group activity recognition algorithm proposed in this paper is effective with low computational cost.