A survey and taxonomy of loss functions in machine learning

Ciampiconi, Lorenzo; Elwood, A.; Leonardi, Marco; Mohamed, Ashraf; Rozza, Alessandro

doi:10.48550/arxiv.2301.05579

Cited by 4 publications

(4 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In detection head network, the loss function [33] is mainly divided into three types: classification loss function, bounding box regression loss function, and confidence loss function. The binary cross-entropy loss function is used for both classification loss function and confidence loss function to complete the classification of objects and calculation of object confidence values.…”

Section: Bounding Box Regression Loss Functionmentioning

confidence: 99%

DMS-YOLOv5: A Decoupled Multi-Scale YOLOv5 Method for Small Object Detection

2023

View full text Add to dashboard Cite

Small objects detection is a challenging task in computer vision due to the limited semantic information that can be extracted and the susceptibility to background interference. In this paper, we propose a decoupled multi-scale small object detection algorithm named DMS-YOLOv5. The algorithm incorporates a receptive field module into the feature extraction network for better focus on low-resolution small objects. The coordinate attention mechanism, which combines spatial and channel attention information, is introduced to reduce interference from background information and enhance the network’s attention to object information. A detection layer tailored to small-sized objects is added to compensate for the loss of small object information in multiple downsampling operations, greatly improving the detection capability of small objects. Next, The decoupled network is introduced into the detection head network for branch processing of classification and bounding box regression tasks. Finally, the bounding box loss function is improved to alleviate missed detection problems caused by the concentration of small objects and mutual occlusion between objects. The improved method achieved a mean average precision improvement of 12.1% on VisDrone2019-DET dataset compared to the original method. In comparison experiments with similar methods, our proposed method also demonstrated good performance, validating its effectiveness.

show abstract

Section: Bounding Box Regression Loss Functionmentioning

confidence: 99%

DMS-YOLOv5: A Decoupled Multi-Scale YOLOv5 Method for Small Object Detection

2023

View full text Add to dashboard Cite

show abstract

“…In energy forecasting, loss optimization is one of the important steps to improve the accuracy of the model [39]. One common technique for loss optimization is using the Adam optimizer, which is a stochastic gradient descent optimizer that uses moving averages of the parameters to adapt the learning rate.…”

Section: Forecasting Model-multivariate Multilayered Lstmmentioning

confidence: 99%

Predicting Commercial Building Energy Consumption Using a Multivariate Multilayered Long-Short Term Memory Time-Series Model

et al. 2023

View full text Add to dashboard Cite

The global demand for energy has been steadily increasing due to population growth, urbanization, and industrialization. Numerous researchers worldwide are striving to create precise forecasting models for predicting energy consumption to manage supply and demand effectively. In this research, a time-series forecasting model based on multivariate multilayered long short-term memory (LSTM) is proposed for forecasting energy consumption and tested using data obtained from commercial buildings in Melbourne, Australia: the Advanced Technologies Center, Advanced Manufacturing and Design Center, and Knox Innovation, Opportunity, and Sustainability Center buildings. This research specifically identifies the best forecasting method for subtropical conditions and evaluates its performance by comparing it with the most commonly used methods at present, including LSTM, bidirectional LSTM, and linear regression. The proposed multivariate, multilayered LSTM model was assessed by comparing mean average error (MAE), root-mean-square error (RMSE), and mean absolute percentage error (MAPE) values with and without labeled time. Results indicate that the proposed model exhibits optimal performance with improved precision and accuracy. Specifically, the proposed LSTM model achieved a decrease in MAE of 30%, RMSE of 25%, and MAPE of 20% compared with the LSTM method. Moreover, it outperformed the bidirectional LSTM method with a reduction in MAE of 10%, RMSE of 20%, and MAPE of 18%. Furthermore, the proposed model surpassed linear regression with a decrease in MAE by 2%, RMSE by 7%, and MAPE by 10%.These findings highlight the significant performance increase achieved by the proposed multivariate multilayered LSTM model in energy consumption forecasting.

show abstract

“…the difference between the observed values y against the predicted values ŷ. A survey by Ciampiconi et al [16] provides a basic overview of regression loss functions. The Mean Bias Error (MBE) loss is considered the pillar from which metrics such as Mean Squared Error (MSE), Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) were developed [16].…”

Section: Fitness Functionmentioning

confidence: 99%

“…A survey by Ciampiconi et al [16] provides a basic overview of regression loss functions. The Mean Bias Error (MBE) loss is considered the pillar from which metrics such as Mean Squared Error (MSE), Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) were developed [16]. Depending on the requirements of the regression task, appropriate metrics are used to evaluate the regression model.…”

Section: Fitness Functionmentioning

confidence: 99%

Multi-target Genetic Programming for Nutrient Assessment in Fish

Hussain

View full text Add to dashboard Cite

<p><strong>The Cyber-Marine project aims to maximise the utility of commercial fish by using Artificial Intelligence (AI) supported tools and non-invasive chemical methods to assess their nutrient content. Vibrational Spectroscopy has proven to be a reliable method to determine molecular characteristics of a product. Particularly, it has gained prominence in evaluating the nutrient level of produce through statistical modelling techniques, such as Partial Least Squared regression (PLSR). This thesis investigates the use of Machine Learning (ML) to relate spectral data of fish with their nutrient levels. Within ML algorithms, Genetic Programming for symbolic regression (GPSR) emerges as a notable approach, producing models that exhibit generalisability and interpretability. As an extension of GPSR, multi-tree GP (MTGP), provides a platform that can simultaneously train models for several nutrients or bioactive components. In this thesis, significant enhancements and modifications are made to key aspects of MTGP, including its evolutionary process, by the introduction of new genetic operator, and the evaluation function, through the use of a distance metric. These are designed not only to leverage the relationship between the spectral information and bioactive components, but also to optimise the interplay between different target variables thereby maximising the capabilities of the algorithm.</strong></p>

show abstract

A survey and taxonomy of loss functions in machine learning

Cited by 4 publications

References 59 publications

DMS-YOLOv5: A Decoupled Multi-Scale YOLOv5 Method for Small Object Detection

DMS-YOLOv5: A Decoupled Multi-Scale YOLOv5 Method for Small Object Detection

Predicting Commercial Building Energy Consumption Using a Multivariate Multilayered Long-Short Term Memory Time-Series Model

Multi-target Genetic Programming for Nutrient Assessment in Fish

Contact Info

Product

Resources

About