Classification of Rare Building Change Using CNN with Multi-Class Focal Loss

Nemoto, Keisuke; Hamaguchi, Ryuhei; Imaizumi, Tsutomu; Hikosaka, Shuhei

doi:10.1109/igarss.2018.8517563

Cited by 25 publications

(12 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The focal loss, with γ = 0 , reduces to the standard cross entropy loss, suggesting that the CE Fig. 4 Focal loss and building change detection [103] loss outperforms the focal loss on two of the three classes in this experiment. To better understand the effectiveness of FL, future work should include a baseline with consistent training data, several alternative methods for addressing class imbalance, and additional performance metrics.…”

Section: Focal Lossmentioning

confidence: 71%

“…Nemoto et al [103] later used the focal loss in another image classification task, the automated detection of rare building changes, e.g. new construction.…”

Section: Focal Lossmentioning

confidence: 99%

“…MFE and MSFE loss functions [18] are modifications of the MSE loss that allow the positive and negative class to equally contribute to the loss. The focal loss function [88,103] improves classification by down-weighting easy-to-classify samples, preventing easily-classified negative samples from dominating the loss. Wang et al modified the CE loss slightly to incorporate pre-defined class costs into the learning process, creating an optimization process that minimizes total cost.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Survey on deep learning with class imbalance

2019

View full text Add to dashboard Cite

Supervised learning methods require labeled training data, and in classification problems each data sample belongs to a known class, or category [1, 2]. In a binary classification problem with data samples from two groups, class imbalance occurs when one class, the minority group, contains significantly fewer samples than the other class, the majority group. In many problems [3-7], the minority group is the class of interest, i.e., the positive class. A well-known class imbalanced machine learning scenario is the medical diagnosis task of detecting disease, where the majority of the patients are

show abstract

Section: Focal Lossmentioning

confidence: 71%

“…Nemoto et al [103] later used the focal loss in another image classification task, the automated detection of rare building changes, e.g. new construction.…”

Section: Focal Lossmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Survey on deep learning with class imbalance

2019

View full text Add to dashboard Cite

show abstract

“…problems, but also hard-sample problems. Nemoto et al [66] later used the FL for the automated detection of rare building changes, e.g. new construction, and concluded that FL improves problems related to class imbalance and over-fitting.…”

Section: Algorithm-level Methodsmentioning

confidence: 99%

Medicare fraud detection using neural networks

Johnson

Khoshgoftaar

2019

J Big Data

View full text Add to dashboard Cite

Access to affordable healthcare is a nationwide concern that impacts a large majority of the United States population. Medicare is a Federal Government healthcare program that provides affordable health insurance to the elderly population and individuals with select disabilities. Unfortunately, there is a significant amount of fraud, waste, and abuse within the Medicare system that costs taxpayers billions of dollars and puts beneficiaries' health and welfare at risk. Previous work has shown that publicly available Medicare claims data can be leveraged to construct machine learning models capable of automating fraud detection, but challenges associated with class-imbalanced big data hinder performance. With a minority class size of 0.03% and an opportunity to improve existing results, we use the Medicare fraud detection task to compare six deep learning methods designed to address the class imbalance problem. Data-level techniques used in this study include random over-sampling (ROS), random under-sampling (RUS), and a hybrid ROS-RUS. The algorithm-level techniques evaluated include a cost-sensitive loss function, the Focal Loss, and the Mean False Error Loss. A range of class ratios are tested by varying sample rates and desirable class-wise performance is achieved by identifying optimal decision thresholds for each model. Neural networks are evaluated on a 20% holdout test set, and results are reported using the area under the receiver operating characteristic curve (AUC). Results show that ROS and ROS-RUS perform significantly better than baseline and algorithm-level methods with average AUC scores of 0.8505 and 0.8509, while ROS-RUS maximizes efficiency with a 4× speedup in training time. Plain RUS outperforms baseline methods with up to 30× improvements in training time, and all algorithm-level methods are found to produce more stable decision boundaries than baseline methods. Thresholding results suggest that the decision threshold always be optimized using a validation set, as we observe a strong linear relationship between the minority class size and the optimal threshold. To the best of our knowledge, this is the first study to compare multiple data-level and algorithm-level deep learning methods across a range of class distributions. Additional contributions include a unique analysis of the relationship between minority class size and optimal decision threshold and state-of-the-art performance on the given Medicare fraud detection task.

show abstract

“…The focal loss reshapes the cross‐entropy loss based on a modulating factor and a hyperparameter to adjust the rate of down‐weighting. However, when applying focal loss on multiclass classification, the results were somewhat contradictory (Nemoto, Hamaguchi, Imaizumi, & Hikosaka, 2018). Besides, the settings of the modulating factor and the hyperparameters are also based on manual experience or multiple times of testing, which further reduces the validity of the algorithm.…”

Section: Literature Reviewmentioning

confidence: 99%

Façade defects classification from imbalanced dataset using meta learning‐based convolutional neural network

Guo

Wang

et al. 2020

Computer aided Civil Eng

View full text Add to dashboard Cite

Façade inspection is a regular but necessary maintenance task to ensure the safety, functioning, and aesthetics of a building. Traditional visual identification of façade defects is dangerous, time-consuming, and insufficient. Based on an image dataset and deep learning algorithms, an automatic façade defects classification technique is developed in this research. A layer-based categorization rule is proposed to categorize façade defects. To handle the problem of imbalanced data size among defect classes, a meta learning-based method is applied, which reassigns weights to the training data. Experiments demonstrated that the proposed method had a stronger capacity to deal with the imbalanced dataset problem comparing with previous methods by improving the classification accuracy from 71.43% of a basic convolutional neural network (CNN) model to 82.86% of a meta learning-based CNN model.

show abstract

Classification of Rare Building Change Using CNN with Multi-Class Focal Loss

Cited by 25 publications

References 6 publications

Survey on deep learning with class imbalance

Survey on deep learning with class imbalance

Medicare fraud detection using neural networks

Façade defects classification from imbalanced dataset using meta learning‐based convolutional neural network

Contact Info

Product

Resources

About