We present an algorithm for video scene detection based on the identification of recurring motion patterns within a video stream. The motion information is extracted in the compressed domain of H.264/AVC videos, no full decoding of the video stream is needed. Based on the motion information our algorithm identifies sequences of adjacent frames with similar motion. Throughout all identified motion sequences we are searching for recurring patterns. The pattern recurring most often is used for the segmentation of the video stream into scenes. The evaluation shows promising results.