Learning Rare Category Classifiers on a Tight Labeling Budget

Mullapudi, Ravi Teja; Poms, Fait; Mark, William R.; Ramanan, Deva; Fatahalian, Kayvon

doi:10.1109/iccv48922.2021.00831

Cited by 4 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternate solution to the problem of imbalanced training data is to curate a more balanced dataset by actively scanning and labelling specimens for the infrequent pollen types (Mullapudi et al, 2021) or to incorporate additional examples from reference pollen slides. Reference pollen has the additional benefit of having verified taxonomic classifications and is subject to fewer annotation errors than the manual labels used in this study.…”

Section: Accuracymentioning

confidence: 99%

Automated identification of diverse Neotropical pollen samples using convolutional neural networks

et al. 2022

View full text Add to dashboard Cite

1. Pollen is used to investigate a diverse range of ecological problems, from identifying plant-pollinator relationships to tracking flowering phenology. Pollen types are identified according to a set of distinctive morphological characters which are understood to capture taxonomic differences and phylogenetic relationships among taxa. However, categorizing morphological variation among hyperdiverse pollen samples represents a challenge even for an expert analyst.2. We present an automated workflow for pollen analysis, from the automated scanning of pollen sample slides to the automated detection and identification of pollen taxa using convolutional neural networks (CNNs). We analysed aerial pollen samples from lowland Panama and used a microscope slide scanner to capture three-dimensional representations of 150 sample slides. These pollen sample images were annotated by an expert using a virtual microscope.Metadata were digitally recorded for ~100 pollen grains per slide, including location, identification and the analyst's confidence of the given identification. We used these annotated images to train and test our detection and classification CNN models. Our approach is two-part. We first compared three methods for training CNN models to detect pollen grains on a palynological slide. We next investigated approaches to training CNN models for pollen identification.3. Because the diversity of pollen taxa in environmental and palaeontological samples follows a long-tailed distribution, we experimented with methods for addressing imbalanced representation using our most abundant 46 taxa. We found that properly weighting pollen taxa in our training objective functions yielded improved accuracy for individual taxa. Our average accuracy for the 46-way classification problem was 82.3%. We achieved 89.5% accuracy for our 25 most abundant taxa. 4. Pollen represents a challenging visual classification problem that can serve as a model for other areas of biology that rely on visual identification. Our results add to the body of research demonstrating the potential for a fully automated [Corrections added on 05 July, 2022, after first online publication: The affiliation of the author "Shu Kong" has been corrected.] | 2051 Methods in Ecology and Evoluঞon PUNYASENA Et Al.

show abstract

Section: Accuracymentioning

confidence: 99%

Automated identification of diverse Neotropical pollen samples using convolutional neural networks

et al. 2022

View full text Add to dashboard Cite

show abstract

“…For example, in financial transaction networks [5], only a small portion of transactions are fraudulent, but it can lead to immeasurable financial loss; in network security [41], identifying malicious activities from large amounts of network traffic can better protect users from potential threats; in patient-symptom network [13], identifying and forecasting rare diseases (i.e., the ones with very few records but severe symptoms) has become a longstanding research problem. Rare category characterization [22] refers to the problem of "finding needles from the hay", which aims to characterize the support regions of minority classes (e.g., needles) from the overwhelmed majority classes (e.g., hay).…”

Section: Introductionmentioning

confidence: 99%

Towards Reliable Rare Category Analysis on Graphs via Individual Calibration

Lei

et al. 2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Rare categories abound in a number of real-world networks and play a pivotal role in a variety of high-stakes applications, including financial fraud detection, network intrusion detection, and rare disease diagnosis. Rare category analysis (RCA) refers to the task of detecting, characterizing, and comprehending the behaviors of minority classes in a highly-imbalanced data distribution. While the vast majority of existing work on RCA has focused on improving the prediction performance, a few fundamental research questions heretofore have received little attention and are less explored: How confident or uncertain is a prediction model in rare category analysis? How can we quantify the uncertainty in the learning process and enable reliable rare category analysis?To answer these questions, we start by investigating miscalibration in existing RCA methods. Empirical results reveal that stateof-the-art RCA methods are mainly over-confident in predicting minority classes and under-confident in predicting majority classes. Motivated by the observation, we propose a novel individual calibration framework, named CaliRare, for alleviating the unique challenges of RCA, thus enabling reliable rare category analysis. In particular, to quantify the uncertainties in RCA, we develop a node-level uncertainty quantification algorithm to model the overlapping support regions with high uncertainty; to handle the rarity of minority classes in miscalibration calculation, we generalize the distribution-based calibration metric to the instance level and propose the first individual calibration measurement on graphs named Expected Individual Calibration Error (EICE). We perform extensive experimental evaluations on real-world datasets, including rare category characterization and model calibration tasks, which demonstrate the significance of our proposed framework.

show abstract

Towards Professional Level Crowd Annotation of Expert Domain Data

Wang,

Vasconcelos

2023

2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

Learning Rare Category Classifiers on a Tight Labeling Budget

Cited by 4 publications

References 19 publications

Automated identification of diverse Neotropical pollen samples using convolutional neural networks

Automated identification of diverse Neotropical pollen samples using convolutional neural networks

Towards Reliable Rare Category Analysis on Graphs via Individual Calibration

Towards Professional Level Crowd Annotation of Expert Domain Data

Contact Info

Product

Resources

About