Linear calibrations in chromatography: The incorrect use of ordinary least squares for determinations at low levels, and the need to redefine the limit of quantification with this regression model

Abstract

Non-standard abbreviations: CV coefficient of variation LOF lack-of-fit OLS ordinary least squares RSD relative standard deviation RSD r relative standard deviation under repeatability conditions PRSD R predicted relative standard deviation of reproducibility s bl standard deviation of the blank TI tolerance interval WLS weighted least squares

The inadequate use of the determination coefficient in analytical calibrations: How other parameters can assess the goodness‐of‐fit more adequately

“…Although it has been extensively demonstrated that OLS should not be applied as the regression model when samples are expected to be determined close to the LOQ [1,[6][7][8][9]12,13,15,[17][18][19], this function is still the most widely applied in all types of laboratories. This fact can mainly be associated with two factors.…”

“…The most used linear regression function, OLS, is based on the minimization of the SSE term (also known as residuals) because this model was developed with the assumption that absolute errors of the dependent variable (measured as SD or variance, SD 2 ) are constant all along the range studied (homoscedasticity). However, in analytical and bioanalytical calibrations the most common situation is that absolute errors are not constant (heteroscedasticity) and the parameter that remains approximately constant is the relative error (RE; RSD) [6,[8][9][10][11][12][13]. In this situation, OLS regression overestimates the effect of calibrators at high concentration ranges, and the higher variations at this level have a much greater influence on R 2 than small deviations present at low ranges [14].…”

“…It has been reported that when the data have a proportional error, neglect of weighting can increase the uncertainty by a factor ≥10 at the lower concentration level [15,16]. For these reasons, it has been reported that OLS should not be used when samples are expected to be determined close to the LOQs [1,[6][7][8][9]12,13,15,[17][18][19], as is very common in trace analysis.…”

“…In this situation, the lower concentration standards become more important. In all the studies that have compared OLS with WLS regression with heteroscedastic calibrations, it has been found that significantly biased results with OLS regression were obtained at low calibration levels [1,12,13,17,20], which are solved applying WLS. However, at a certain level above the LOQ of the method, both linear regression models tend to yield equivalent results.…”

Principal component regression that minimizes the sum of the squares of the relative errors: Application in multivariate calibration models

“…Concerning univariate calibration, linear regression by LS has been used with the response weighted by the variance of the experimental signal. 5 Recently, [6][7][8] different weights have been suggested (1/y 2 , 1/y,1/y 0.5 ,1/x 2 ,1/x,1/x 0.5 , x being the concentration and y the signal) aiming at selecting the one that best fit the data. Nevertheless, this does not result in the univariate version of LSRE, since relative errors are not those of the concentration but those of the experimental signal.…”