Hypothesis-driven Online Video Stream Learning with Augmented Memory

Zhang, Mengmi; Badkundri, Rohil; Talbot, Morgan B.; Zawar, Rushikesh; Kreiman, Gabriel

doi:10.48550/arxiv.2104.02206

Cited by 1 publication

(1 citation statement)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, Incremental is a lower bound among all computational models. Previous works have shown weight-regularization methods are less effective than replay methods in many continual learning tasks [30,29], we included the weight-regularization method EWC for comparison. We observed that EWC is inferior to other competitive baselines in stream learning.…”

Section: Performance In Fully Supervised Protocolmentioning

confidence: 99%

Label-Efficient Online Continual Object Detection in Streaming Video

Wu¹,

Zhang²,

Hsu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

To thrive in evolving environments, humans are capable of continual acquisition and transfer of new knowledge, from a continuous video stream, with minimal supervisions, while retaining previously learnt experiences. In contrast to human learning, most standard continual learning benchmarks focus on learning from static iid images in fully supervised settings. Here, we examine a more realistic and challenging problem-Label-Efficient Online Continual Object Detection (LEOCOD) in video streams. By addressing this problem, it would greatly benefit many real-world applications with reduced data annotation costs and model retraining time. To tackle this problem, we seek inspirations from complementary learning systems (CLS) in human brains and propose a computational model, dubbed as Efficient-CLS. Functionally correlated with the hippocampus and the neocortex in CLS, Efficient-CLS posits a memory encoding mechanism involving bidirectional interaction between fast and slow learners via synaptic weight transfers and pattern replays. We test Efficient-CLS and competitive baselines in two challenging real-world video stream datasets. Like humans, Efficient-CLS learns to detect new object classes incrementally from a continuous temporal stream of non-repeating video with minimal forgetting. Remarkably, with only 25% annotated video frames, our Efficient-CLS still leads among all comparative models, which are trained with 100% annotations on all video frames. The data and source code will be publicly available at https://github.com/showlab/Efficient-CLS.

show abstract

Section: Performance In Fully Supervised Protocolmentioning

confidence: 99%