A video indexing and retrieval computational prototype based on transcribed speech

Spolaôr, Newton; Lee, Huei; Takaki, Weber Shoity Resende; Ensina, Leandro Augusto; Parmezan, Antonio Rafael Sabino; Oliva, Jefferson Tales; Coy, Cláudio Saddy Rodrigues; Wu, Feng

doi:10.1007/s11042-021-11401-1

Cited by 6 publications

(3 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…vitrivr is a "a scalable, content-based multimedia information retrieval stack" [3]. Its general purpose nature allows usage in video retrieval [9,35], cultural heritage [22,30], and it is a regular participant at LSC [11,14,23] with its core architecture unchanged in the last few iterations. The system has three core components: a database specialized for multimedia retrieval, Cottontail DB [2], a retrieval engine, Cineast [27], and the web-based user interface, vitrivr-ng.…”

Section: Vitrivrmentioning

confidence: 99%

vitrivr at the Lifelog Search Challenge 2022

Heller

Rossetto

Sauter

et al. 2022

Proceedings of the 5th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

show abstract

Section: Vitrivrmentioning

confidence: 99%

vitrivr at the Lifelog Search Challenge 2022

Heller

Rossetto

Sauter

et al. 2022

Proceedings of the 5th Annual on Lifelog Search Challenge

View full text Add to dashboard Cite

show abstract

“…Presently, the issue of poor generalizability in the face of scarcely available labelled medical data is addressed by utilizing transfer learning techniques and data augmentation [10]. Initializing model weights using datasets such as ImageNet [11] leverages learned features to reduce inference times and improve generalizability.…”

Section: Introductionmentioning

confidence: 99%

Improving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data for Lung Sliding Artifact Identification

Wu,

Smith,

VanBerlo

et al. 2024

Diagnostics

View full text Add to dashboard Cite

Deep learning (DL) models for medical image classification frequently struggle to generalize to data from outside institutions. Additional clinical data are also rarely collected to comprehensively assess and understand model performance amongst subgroups. Following the development of a single-center model to identify the lung sliding artifact on lung ultrasound (LUS), we pursued a validation strategy using external LUS data. As annotated LUS data are relatively scarce—compared to other medical imaging data—we adopted a novel technique to optimize the use of limited external data to improve model generalizability. Externally acquired LUS data from three tertiary care centers, totaling 641 clips from 238 patients, were used to assess the baseline generalizability of our lung sliding model. We then employed our novel Threshold-Aware Accumulative Fine-Tuning (TAAFT) method to fine-tune the baseline model and determine the minimum amount of data required to achieve predefined performance goals. A subgroup analysis was also performed and Grad-CAM++ explanations were examined. The final model was fine-tuned on one-third of the external dataset to achieve 0.917 sensitivity, 0.817 specificity, and 0.920 area under the receiver operator characteristic curve (AUC) on the external validation dataset, exceeding our predefined performance goals. Subgroup analyses identified LUS characteristics that most greatly challenged the model’s performance. Grad-CAM++ saliency maps highlighted clinically relevant regions on M-mode images. We report a multicenter study that exploits limited available external data to improve the generalizability and performance of our lung sliding model while identifying poorly performing subgroups to inform future iterative improvements. This approach may contribute to efficiencies for DL researchers working with smaller quantities of external validation data.

show abstract

“…2 Both systems use the same retrieval engine [ 30 ] and database [ 4 ], but offer different query and interaction methods. vitrivr-VR builds upon the vitrivr system, which has been used in a variety of contexts such as cultural heritage [ 20 , 22 , 34 ], retrieval for speech transcription [ 39 ], and video retrieval [ 11 ].…”

Section: Introductionmentioning

confidence: 99%

A tale of two interfaces: vitrivr at the lifelog search challenge

Heller

Spieß

Schuldt

2023

Multimed Tools Appl

View full text Add to dashboard Cite

The past decades have seen an exponential growth in the amount of data which is produced by individuals. Smartphones which capture images, videos and sensor data have become commonplace, and wearables for fitness and health are growing in popularity. Lifelog retrieval systems aim to aid users in finding and exploring their personal history. We present two systems for lifelog retrieval: vitrivr and vitrivr-VR, which share a common retrieval model and backend for multi-modal multimedia retrieval. They differ in the user interface component, where vitrivr relies on a traditional desktop-based user interface and vitrivr-VR has a Virtual Reality user interface. Their effectiveness is evaluated at the Lifelog Search Challenge 2021, which offers an opportunity for interactive retrieval systems to compete with a focus on textual descriptions of past events. Our results show that the conventional user interface outperformed the VR user interface. However, the format of the evaluation campaign does not provide enough data for a thorough assessment and thus to make robust statements about the difference between the systems. Thus, we conclude by making suggestions for future interactive evaluation campaigns which would enable further insights.

show abstract

A video indexing and retrieval computational prototype based on transcribed speech

Cited by 6 publications

References 74 publications

vitrivr at the Lifelog Search Challenge 2022

vitrivr at the Lifelog Search Challenge 2022

Improving the Generalizability and Performance of an Ultrasound Deep Learning Model Using Limited Multicenter Data for Lung Sliding Artifact Identification

A tale of two interfaces: vitrivr at the lifelog search challenge

Contact Info

Product

Resources

About