An optimization model for temporal video lecture segmentation using word2vec and acoustic features

Soares, Eduardo R.; Barrére, Eduardo

doi:10.1145/3323503.3349548

Cited by 10 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Many recent works analyze the audio or audio transcripts of lecture videos with different objectives such as topic-wise segmentation of the lecture. The video is usually divided into small segments (e.g by using voice activity detection [38]) or temporal windows. Then, existing transcripts or automatic speech recognition (ASR) are used to create an embedding for each segment based on methods like bags of words [39], word2vec [38]- [40], and TF-IDF [41].…”

Section: ) Audio-based Analysismentioning

confidence: 99%

“…The video is usually divided into small segments (e.g by using voice activity detection [38]) or temporal windows. Then, existing transcripts or automatic speech recognition (ASR) are used to create an embedding for each segment based on methods like bags of words [39], word2vec [38]- [40], and TF-IDF [41]. Some methods also consider additional acoustic features [38], [40].…”

Section: ) Audio-based Analysismentioning

confidence: 99%

See 1 more Smart Citation

FCN-LectureNet: Extractive Summarization of Whiteboard and Chalkboard Lecture Videos

Davila

Setlur

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Recording and sharing of educational or lecture videos has increased in recent years. Within these recordings, we find a large number of math-oriented lectures and tutorials which attract students of all levels. Many of the topics covered by these recordings are better explained using handwritten content on whiteboards or chalkboards. Hence, we find large numbers of lecture videos that feature the instructor writing on a surface. In this work, we propose a novel method for extraction and summarization of the handwritten content found in such videos. Our method is based on a fully convolutional network, FCN-LectureNet, which can extract the handwritten content from the video as binary images. These are further analyzed to identify the unique and stable units of content to produce a spatial-temporal index of handwritten content. A signal which approximates content deletion events is then built using information from the spatial-temporal index. The peaks of this signal are used to create temporal segments of the lecture based on the notion that sub-topics change when large portions of content are deleted. Finally, we use these segments to create an extractive summary of the handwritten content based on key-frames. This will facilitate content-based search and retrieval of these lecture videos. In this work, we also extend the AccessMath dataset to create a novel dataset for benchmarking of lecture video summarization called LectureMath. Our experiments on both datasets show that our novel method can outperform existing methods especially on the larger and more challenging dataset. Our code and data will be made publicly available.INDEX TERMS fully convolutional networks, handwritten text detection, image binarization, lecture videos, video summarization.

show abstract

Section: ) Audio-based Analysismentioning

confidence: 99%

Section: ) Audio-based Analysismentioning

confidence: 99%

FCN-LectureNet: Extractive Summarization of Whiteboard and Chalkboard Lecture Videos

Davila

Setlur

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Em relação aos módulos, todos são embarcados em contêineres Docker 1 para facilitar a reprodutibilidade e a implantação de nossa proposta. Para implementar o algoritmo proposto por [Soares and Barrére 2019] foi necessária a instanciação de 7 módulos. A Figura 3 ilustra o esquema de comunicação em fila entre os módulos de arquitetura.…”

Section: Implementaçãounclassified

“…Como prova de conceito, implementamos o fluxo de um algoritmo de segmentação baseado em fala proposto anteriormente em [Soares and Barrére 2019] utilizando o Easy-Topic. Também, comparamos os resultados obtidos por nossa implementação com outros dois frameworks da literatura.…”

Section: Conclusão E Trabalhos Futurosunclassified

“…Embora existam muitas abordagens na literatura para automatizar essa tarefa, a maioria delas se baseia em recursos de materiais complementares, como slides, livros didáticos e legendas; tornando essas abordagens muito dependentes da disponibilidade de recursos específicos, dificultando assim o uso dessas abordagens em boa parte das videoaulas disponíveis na internet. Visando minimizar essas limitações, desenvolvemos uma solução baseada no discurso do professor para a segmentação temporal em tópicos de qualquer videoaula [Soares and Barrére 2019] que possua ou não recursos adicionais. Para dar suporte, flexibilidade e escalabilidade a essa solução, foi desenvolvida uma solução que permite incorporar módulos para obtenção de diversas características prosódicas e semânticas do vídeo e também algoritmos de segmentação diversos.…”

Section: Introductionunclassified

See 1 more Smart Citation

Framework para Segmentação Temporal de Vídeos Educacionais

Barrére¹,

Souza²,

Soares³

2020

Anais Do XXXI Simpósio Brasileiro De Informática Na Educação (SBIE 2020)

Self Cite

View full text Add to dashboard Cite

Este artigo discute os principais aspectos relacionados aos frameworks de segmentação de vídeos educacionais (videoaulas). Apresenta um framework, denominado EasyTopic, que permite a utilização de diversas abordagens para o processo de segmentação temporal de vídeoaulas em tópicos (segmentos semânticos), de forma configurável e generalista. Com o objetivo de comprovar o funcionamento do framework, apresenta seu uso na implementação de uma solução para o problema de segmentação em tópicos de vídeos educacionais, com resultados e considerações sobre o uso do framework.

show abstract

Skeleton-Based Methods for Speaker Action Classification on Lecture Videos

Davila

Setlur

et al. 2021

Pattern Recognition. ICPR International Workshops and Challenges

View full text Add to dashboard Cite

An optimization model for temporal video lecture segmentation using word2vec and acoustic features

Cited by 10 publications

References 19 publications

FCN-LectureNet: Extractive Summarization of Whiteboard and Chalkboard Lecture Videos

FCN-LectureNet: Extractive Summarization of Whiteboard and Chalkboard Lecture Videos

Framework para Segmentação Temporal de Vídeos Educacionais

Skeleton-Based Methods for Speaker Action Classification on Lecture Videos

Contact Info

Product

Resources

About