Two Statistical Approaches to Justify the Use of the Logistic Function in Binary Logistic Regression

Abstract

Logistic regression is a commonly used classification algorithm in machine learning. It allows categorizing data into discrete classes by learning the relationship from a given set of labeled data. It learns a linear relationship from the given dataset and then introduces nonlinearity through an activation function to determine a hyperplane that separates the learning points into two subclasses. In the case of logistic regression, the logistic function is the most used activation function to perform binary classification. The choice of logistic function for binary classifications is justified by its ability to transform any real number into a probability between 0 and 1. This study provides, through two different approaches, a rigorous statistical answer to the crucial question that torments us, namely where does this logistic function on which most neural network algorithms are based come from? Moreover, it determines the computational cost of logistic regression, using theoretical and experimental approaches.

Predicting wildfires in Algerian forests using machine learning models

“…For a given observation , the random variable follows Bernoulli distribution with parameter , i.e., , and where designates the probability of the event , and is a mapping from to called score function [ 22 ]. Given a threshold , we decide that: …”

Recognition and Categorization of Blood Groups by Machine Learning and Image Processing Method

“…The sigmoid function, utilized in logistic regression, can be mathematically expressed by Equation below. This process aims to transform the actual values of independent variables into a range of 0-1; this is done to convert the continuous output values of the linear regression function into categorical values [39,40]. It can be represented using Equations below:…”

A comparative study of heterogeneous machine learning algorithms for arrhythmia classification using feature selection technique and multi-dimensional datasets

“…For example, if the output of the sigmoid function is greater than 0.5, the input sample is assigned to class 1. Otherwise, it is assigned to class 0 [26]. The objective function of LR is:…”