Influence of feature scaling on convergence of gradient iterative algorithm

Abstract

Feature scaling is a method to unify self-variables or feature ranges in data. In data processing, it is usually used in data pre-processing. Because in the original data, the range of variables is very different. Feature scaling is a necessary step in the calculation of stochastic gradient descent. This paper takes the computer hardware data set maintained by UCI as an example, and compares the influence of normalization method and interval scaling method on the convergence of stochastic gradient descent by algorithm simulation. The result of study has a certain value on feature scaling.

Human Activity Recognition for Indoor Localization Using Smartphone Inertial Sensors

“…The purpose of performing this scaling step was to standardize the independent feature values present in the data to the same fixed range. This helped in the handling of highly varying magnitudes, values, or units present in the input data [ 29 ]. Moreover, when using gradient descent optimization, the presence of a higher feature value in the sample affects how the step size of the gradient descent changes, causing different step sizes for each feature, which may slow the gradient convergence during training [ 29 ].…”

“…This helped in the handling of highly varying magnitudes, values, or units present in the input data [ 29 ]. Moreover, when using gradient descent optimization, the presence of a higher feature value in the sample affects how the step size of the gradient descent changes, causing different step sizes for each feature, which may slow the gradient convergence during training [ 29 ]. Thus, a standardization technique based on the mean and standard deviation of the features values was applied to re-scale each independent feature value so that it had a distribution with zero mean and unit variance.…”

Online Measuring of Robot Positions Using Inertial Measurement Units, Sensor Fusion and Artificial Intelligence

“…As discussed earlier, the measurements taken by the sensors are not accurate, and noise tends to increase over time. One way to place the values within the same range is through the technique called Feature Scaling [38], which facilitates the operation of most neural networks (normalize independent variables on a specific scale. As the input is noisy, this implies that the range of raw values varies widely and, therefore, hampers the objective function of many networks to function correctly.…”

Büyük Veri Ortamlarında Zararlı Yazılım Tespiti Kapsamında Makine Öğrenmesi Algoritmalarının Performansının İncelenmesi

“…Orijinal verilerde değişkenlerin aralığının çok farklı olması sorununu ortadan kaldırmak için veri önişleme adımında özellik ölçeklendirme yapılır [32]. Özellik vektörünün ölçeklendirilmesi için Pyspark kütüphanesinde StandardScaler metodundan faydalanıldı ve özellikler 0 ile 1 arasında ölçeklendirildi.…”