GAN based efficient foreground extraction and HGWOSA based optimization for video synopsis generation

Ghatak, Subhankar; Rup, Suvendu; Didwania, Himansu; Swamy, M.N.S.

doi:10.1016/j.dsp.2021.102988

Cited by 11 publications

(9 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…18 The former is an improved energy minimization scheme that uses a hybrid algorithm of SA and teaching-learning based optimization algorithm, and the latter is a hybrid algorithm using SA and JAYA algorithm 37 to minimize the energy function. Ghatak et al 23 proposed an optimization algorithm through the hybridization of SA and grey wolf optimizer (GWO), named as, HGWOSA to ensure global optimal result with a low computing overhead.…”

Section: Traditional Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Surveillance video synopsis based on spatio-temporal offset

2023

View full text Add to dashboard Cite

.With the wide popularity of surveillance cameras, video synopsis technology has become a research hotspot. The existing methods of surveillance video synopsis usually summarize the input video by shifting the object tube in the video on the time axis, which ignore the serious collision artifacts and chronological disorder between moving objects. To solve these problems, we propose a surveillance video synopsis methodology called “surveillance video synopsis based on spatio-temporal offset (STO)” that can simultaneously shift the moving object in the temporal domain and spatial domain. First, object detection and tracking algorithms are used to extract the object tube from the input video. Two collision relations are proposed by analyzing relationship between tubes to classify collision artifacts. Then, we present two spatial offset cases to find the optimal spatial offset of the object tube. Finally, an adaptive optimization frame density model is proposed to analyze the optimal temporal offset of the object tube. Simultaneously, the object tube and the background are stitched according to the STO to generate the synopsis video. Extensive experimental results demonstrate the effectiveness of the proposed method in improving frame compression rate and alleviating collision artifacts.

show abstract

Section: Traditional Methodsmentioning

confidence: 99%

“…On the one hand, during the shifting operation of the object tube in the time axis, collision artifact will be generated 22 – 24 On the other hand, the new time label of the object tubes tends to change the chronological order of the objects via events rearrangement optimization, 25 called CE method.…”

Section: Introductionmentioning

confidence: 99%

Surveillance video synopsis based on spatio-temporal offset

2023

View full text Add to dashboard Cite

show abstract

“…Moussa and Shoitan 16 proposed a particle swarm algorithm for optimizing the energy function and experimentally proved that the particle swarm optimization algorithm outperforms the genetic algorithm and achieves the optimal solution of the genetic algorithm at a lower computational cost. Ghatak et al [12][13][14]21 proposed various hybrid algorithms combining simulated annealing with the teach-and-learn algorithm, JAJY algorithm, gray wolf optimization algorithm, and scenario optimization, respectively, to form the new hybrid algorithms HSATLBO, HSAJAYA, HGWOSA, and cSAScO for global energy function minimization.…”

Section: Optimization Algorithm-based Video Synopsismentioning

confidence: 99%

“…Some researchers have combined intelligent optimization algorithms with simulated annealing to form new hybrid algorithms for optimizing global energy functions. Ghatak et al [12][13][14] combined simulated annealing with TLBO, JAJY, and GWO algorithms, respectively. However, current optimization methods suffer from high computational complexity and time-consuming engagement.…”

Section: Introductionmentioning

confidence: 99%

Video synopsis method based on interaction determination using human pose estimation

Wang,

Yang,

Wang

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

Video synopsis aims to condense long videos containing temporal redundancy into a more compact form by rearranging the active objects in the video. However, since each object in the video belongs to a different tube, moving the objects during the tube rearrangement process may disrupt the interaction relationship between the objects. To address this issue, we proposed a video synopsis optimization method based on interaction determination using human pose estimation. First, the inverse perspective mapping method is used to calculate the distance between objects from a bird's-eye view. Then the body orientation is determined using the coordinates of human skeletal points obtained from the human pose estimation model. These two pieces of information are then combined to determine the interactions between objects. Second, the interaction energy cost term is introduced. By penalizing the cost term, the interactions between objects in the original video are preserved in the synopsis video. Finally, we propose a fusion optimization algorithm (FSOSA) that combines snake optimization and simulated annealing. FSOSA aims to minimize the energy cost function and achieve optimal tube rearrangement. Experimental results demonstrate that the proposed method effectively preserves the interactions between objects while improving the convergence speed.

show abstract

“…Therefore, a range tree technique was suggested to select object tubes and reduce the algorithm complexity efficiently. Ghatak et al [ 23 ] explored the notion of the multi-frame and scale procedure together with generative adversarial networks (MFS–GANs) to extract the foreground. A hybrid algorithm, including both grey wolf optimizer (GWO) and SA (HGWOSA), is suggested as an optimization algorithm to achieve the globally optimal result with a low computation cost.…”

Section: Introductionmentioning

confidence: 99%

User Preference-Based Video Synopsis Using Person Appearance and Motion Descriptions

Shoitan

Moussa

Gharghory

et al. 2023

Sensors

View full text Add to dashboard Cite

During the last decade, surveillance cameras have spread quickly; their spread is predicted to increase rapidly in the following years. Therefore, browsing and analyzing these vast amounts of created surveillance videos effectively is vital in surveillance applications. Recently, a video synopsis approach was proposed to reduce the surveillance video duration by rearranging the objects to present them in a portion of time. However, performing a synopsis for all the persons in the video is not efficacious for crowded videos. Different clustering and user-defined query methods are introduced to generate the video synopsis according to general descriptions such as color, size, class, and motion. This work presents a user-defined query synopsis video based on motion descriptions and specific visual appearance features such as gender, age, carrying something, having a baby buggy, and upper and lower clothing color. The proposed method assists the camera monitor in retrieving people who meet certain appearance constraints and people who enter a predefined area or move in a specific direction to generate the video, including a suspected person with specific features. After retrieving the persons, a whale optimization algorithm is applied to arrange these persons reserving chronological order, reducing collisions, and assuring a short synopsis video. The evaluation of the proposed work for the retrieval process in terms of precision, recall, and F1 score ranges from 83% to 100%, while for the video synopsis process, the synopsis video length compared to the original video is decreased by 68% to 93.2%, and the interacting tube pairs are preserved in the synopsis video by 78.6% to 100%.

show abstract

GAN based efficient foreground extraction and HGWOSA based optimization for video synopsis generation

Cited by 11 publications

References 27 publications

Surveillance video synopsis based on spatio-temporal offset

Surveillance video synopsis based on spatio-temporal offset

Video synopsis method based on interaction determination using human pose estimation

User Preference-Based Video Synopsis Using Person Appearance and Motion Descriptions

Contact Info

Product

Resources

About