A New Multipoint Symmetric Secant Method with a Dense Initial Matrix

Abstract

In large-scale optimization, when either forming or storing Hessian matrices are prohibitively expensive, quasi-Newton methods are often used in lieu of Newton's method because they only require first-order information to approximate the true Hessian. Multipoint symmetric secant (MSS) methods can be thought of as generalizations of quasi-Newton methods in that they attempt to impose additional requirements on their approximation of the Hessian. Given an initial Hessian approximation, MSS methods generate a sequence of matrices using rank-2 updates. For practical reasons, up to now, the initialization has been a constant multiple of the identity matrix. In this paper, we propose a new limited-memory MSS method that allows for dense initializations. Numerical results on the CUTEst test problems suggest that the MSS method using a dense initialization outperforms the standard initialization. Numerical results also suggest that this approach is competitive with a basic L-SR1 trust-region method. DedicationThis paper is dedicated to Oleg Burdakov who passed away June 1, 2021. S k = [s k−1 s k−2 . . . s 0 ] and Y k = [y k−1 y k−2 . . . y 0 ] .

A machine learning analysis of COVID-19 mental health data

“…Finally, we would like to identify specific features then utilize generalized linear models to more exactly quantify the relationship the variables have with mental health decline. We are also interested in analyzing accuracy, speed and feature scores of some machine learning models on the COVID-19 mental health data when we replace the most common optimizers such as GD, SGD or Limitedmemory BFGS with the ones that are introduced recently 32,33 .…”

A Machine Learning Analysis of Impact of the Covid-19 Pandemic on Alcohol Consumption Habit Changes Among Healthcare Workers in the U.S.

“…In future work, we are interested in analyzing more data related to healthcare workers to examine the relationship between their mental health and other factors related or unrelated to COVID-19. We are also interested in implementing other optimization methods such as those that are introduced in [55] and [56] to see whether accuracy or speed of some models, which were utilized in this paper, can be improved.…”

Artificial intelligence-based analytics for impacts of COVID-19 and online learning on college students’ mental health

“…In supervised learning, we convert the learning problem into an optimization problem by defining a loss function as the objective function. Optimization methods (e.g., [37][38][39][40][41][42][43][44][45][46]), which are often used to minimize a loss or error function in the model training process, play an important role in the speed-accuracy trade-off of machine learning algorithms. In the remainder of this section, we briefly review all supervised and unsupervised learning methods used in this paper.…”