Video Summarization for Sign Languages Using the Median of Entropy of Mean Frames Method

Abstract

Multimedia information requires large repositories of audio-video data. Retrieval and delivery of video content is a very time-consuming process and is a great challenge for researchers. An efficient approach for faster browsing of large video collections and more efficient content indexing and access is video summarization. Compression of data through extraction of keyframes is a solution to these challenges. A keyframe is a representative frame of the salient features of the video. The output frames must represent the original video in temporal order. The proposed research presents a method of keyframe extraction using the mean of consecutive k frames of video data. A sliding window of size k / 2 is employed to select the frame that matches the median entropy value of the sliding window. This is called the Median of Entropy of Mean Frames (MME) method. MME is mean-based keyframes selection using the median of the entropy of the sliding window. The method was tested for more than 500 videos of sign language gestures and showed satisfactory results.

Entropy in Image Analysis

“…Saqib and Kazmi [ 3 ] propose a solution to the problem of the retrieval and delivery of contents from audio-video repositories, in order to achieve faster browsing of collections. The compression of data is achieved by means of keyframes, which are representative frames of the salient features of the videos.…”

Keyframe Extraction Algorithm for Continuous Sign-Language Videos Using Angular Displacement and Sequence Check Metrics

“…Specifcally, tapping or rubbing does not propagate data over successive frames, preventing static thresholds from distinguishing movements between such frames. Solutions based on threshold values like entropy or sampling do not address scalability or signer independence [14,24]. Tis work handled these sign gestures efectively and consistently throughout the huge dataset, which had never been studied before.…”

“…A hybrid clustering approach is provided in [38] and two sets of keyframes are obtained; the spliced original keyframe picture represents the spatial dimension feature, and the optical fow keyframe image represents the time dimension feature. Te author of [24] proposed the median of entropy of mean frames (MME) approach for keyframe extraction, which uses the mean of consecutive k frames of video data with a sliding window of size k/2 to select the frame that satisfes the median entropy value. Te methodology used in [39] considers multievaluation factors to select critical frames from raw videos.…”