Low-Shot Learning from Imaginary Data

Wang, Yu-Xiong; Girshick, Ross; Hebert, Martial; Hariharan, Bharath

doi:10.1109/cvpr.2018.00760

Cited by 704 publications

(475 citation statements)

References 18 publications

Supporting

Mentioning

473

Contrasting

Unclassified

Order By: Relevance

“…with current ML techniques. However, advances in low‐shot learning (e.g., Wang, Girshick, Hebert, & Hariharan, ) may allow our annotations to be used to create such a fine‐grained classifier in the future.…”

Section: Methodsmentioning

confidence: 99%

CityNet—Deep learning tools for urban ecoacoustic assessment

et al. 2018

View full text Add to dashboard Cite

Cities support unique and valuable ecological communities, but understanding urban wildlife is limited due to the difficulties of assessing biodiversity. Ecoacoustic surveying is a useful way of assessing habitats, where biotic sound measured from audio recordings is used as a proxy for population abundance and/or activity. However, existing algorithms systematically over and underestimate measures of biotic activity in the presence of typical urban non‐biotic sounds in recordings. We develop CityNet, a deep learning system using convolutional neural networks (CNNs), to measure audible biotic (CityBioNet) and anthropogenic (CityAnthroNet) acoustic activity in cities. The CNNs were trained on a large dataset of annotated audio recordings collected across Greater London, UK. Using a held‐out test dataset, we compare the precision and recall of CityBioNet and CityAnthroNet separately to the best available alternative algorithms: four Acoustic Indices: Acoustic Complexity Index, Acoustic Diversity Index, Bioacoustic Index, and Normalised Difference Soundscape Index, and a state‐of‐the‐art bird call detection CNN (bulbul). We also compare the effect of non‐biotic sounds on the predictions of CityBioNet and bulbul. Finally we apply CityNet to describe acoustic patterns of the urban soundscape in two sites along an urbanisation gradient. CityBioNet was the best performing algorithm for measuring biotic activity in terms of precision and recall, followed by bulbul, whereas the Acoustic Indices performed worst. CityAnthroNet outperformed the Normalised Difference Soundscape Index, but by a smaller margin than CityBioNet achieved against the competing algorithms. The CityBioNet predictions were impacted by mechanical sounds, whereas air traffic and wind sounds influenced the bulbul predictions. Across an urbanisation gradient, we show that CityNet produced realistic daily patterns of biotic and anthropogenic acoustic activity from real‐world urban audio data. Using CityNet, it is possible to automatically measure biotic and anthropogenic acoustic activity in cities from audio recordings. If embedded within an autonomous sensing system, CityNet could produce environmental data for cites at large‐scales and facilitate investigation of the impacts of anthropogenic activities on wildlife. The algorithms, code and pretrained models are made freely available in combination with two expert‐annotated urban audio datasets to facilitate automated environmental surveillance in cities.

show abstract

Section: Methodsmentioning

confidence: 99%

CityNet—Deep learning tools for urban ecoacoustic assessment

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Few-shot learning. There is a broad array of few-shot learning approaches, including, among many: gradient descent-based approaches [1,11,38,44], which learn how to rapidly adapt a model to a given few-shot recognition task via a small number of gradient descent iterations; metric learning based approaches that learn a distance metric be-tween a query, i.e., test image, and a set of support images, i.e., training images, of a few-shot task [26,52,54,56,58]; methods learning to map a test example to a class label by accessing memory modules that store training examples for that task [12,25,34,37,49]; approaches that learn how to generate the weights of a classifier [13,16,42,43] or of a multi-layer neural network [3,18,19,57] for the new classes given the few available training data for each of them; methods that "hallucinate" additional examples of a class from a reduced amount of data [20,56].…”

Section: Related Workmentioning

confidence: 99%

Boosting Few-Shot Visual Learning With Self-Supervision

Gidaris

Bursuc

Komodakis

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

379

221

View full text Add to dashboard Cite

Few-shot learning and self-supervised learning address different facets of the same problem: how to train a model with little or no labeled data. Few-shot learning aims for optimization methods and models that can learn efficiently to recognize patterns in the low data regime. Self-supervised learning focuses instead on unlabeled data and looks into it for the supervisory signal to feed high capacity deep neural networks. In this work we exploit the complementarity of these two domains and propose an approach for improving few-shot learning through self-supervision. We use self-supervision as an auxiliary task in a few-shot learning pipeline, enabling feature extractors to learn richer and more transferable visual representations while still using few annotated samples. Through self-supervision, our approach can be naturally extended towards using diverse unlabeled data from other datasets in the few-shot setting. We report consistent improvements across an array of architectures, datasets and self-supervision techniques.

show abstract

“…In terms of a generative model, Wang et al [26] generated data that has similar characteristics to the training examples for novel categories. Similarly, Hariharan and Girshick [27] presented an example generation function where transformations learned from base categories are applied to the examples of novel categories.…”

Section: Related Workmentioning

confidence: 99%

“…To assess the benefit of our finetuning strategy, we show the accuracy when not fine-tuning the network. In this case, the weights of novel categories are [26] AND WE REFERRED TO [19]. Both Both with prior k=1 2 5 10 20 k=1 2 5 10 20 k=1 2 5 10 20…”

Section: Ablation Studymentioning

confidence: 99%

Few-Shot Learning With Geometric Constraints

Jung

Lee

2020

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

In this paper, we consider the problem of few-shot learning for classification. We assume a network trained for base categories with a large number of training examples, and we aim to add novel categories to it that have only a few, e.g., one or five, training examples. This is a challenging scenario because (1) high performance is required in both the base and novel categories, and (2) training the network for the new categories with few training examples can contaminate the feature space trained well for the base categories. To address these challenges, we propose two geometric constraints to fine-tune the network with a few training examples. The first constraint enables features of the novel categories to cluster near the category weights, and the second maintains the weights of the novel categories far from the weights of the base categories. By applying the proposed constraints, we extract discriminative features for the novel categories while preserving the feature space learned for the base categories. Using public datasets for few-shot learning that are subsets of ImageNet, we demonstrate that the proposed method outperforms prevalent methods by a large margin.

show abstract

Low-Shot Learning from Imaginary Data

Cited by 704 publications

References 18 publications

CityNet—Deep learning tools for urban ecoacoustic assessment

CityNet—Deep learning tools for urban ecoacoustic assessment

Boosting Few-Shot Visual Learning With Self-Supervision

Few-Shot Learning With Geometric Constraints

Contact Info

Product

Resources

About