In this paper, we propose a novel method for unusual human activity detection in crowded scenes. Specifically, rather than detecting or segmenting humans, we devised an efficient method, called a motion influence map, for representing human activities. The key feature of the proposed motion influence map is that it effectively reflects the motion characteristics of the movement speed, movement direction, and size of the objects or subjects and their interactions within a frame sequence. Using the proposed motion influence map, we further developed a general framework in which we can detect both global and local unusual activities. Furthermore, thanks to the representational power of the proposed motion influence map, we can localize unusual activities in a simple manner. In our experiments on three public datasets, we compared the performances of the proposed method with that of other state-of-the-art methods, and showed that the proposed method outperforms these competing methods.