Mathematical Model of Logistic Regression for Binary Classification. Part 1. Regression Models of Data Generalization

Abstract

In this article, the mathematical justification of logistic regression as an effective and simple to implement method of machine learning is performed. A review of literary sources was conducted in the direction of statistical processing, analysis and classification of data using the logistic regression method, which confirmed the popularity of this method in various subject areas. The logistic regression method was compared with the linear and probit regression methods regarding the possibility of predicting the probabilities of events. In this context, the disadvantages of linear regression and the advantages and affinity of logit and probit regression methods are noted. It is indicated that the possibility of forecasting probabilities and binary classification by the method of logistic regression is provided by the use of a sigmoid function with the property of compressive transformation of an argument with an unlimited numerical value into a limited range from 0 to 1 real value of the function. The derivation of the sigmoid function in two different ways is described: based on the model of the logarithm of the odds of events and the model of logistic population growth. Based on the method of maximum likelihood, the construction of a logarithmic loss function was demonstrated, the use of which made it possible to move from a multi-extremal nonlinear regression problem to a unimodal optimization problem. Methods of regularization of the loss function are presented to control the complexity and prevent retraining of the logistic regression model.