Quasi-Newton (QN) methods have shown to be effective in training neural networks. However, the computation and the storage of the approximated Hessian in large-scale applications is still a problem. The Memory-less QN (MLQN) was introduced as a method that did not require the storage of the matrix. This paper describes the effectiveness of the momentum term for the accelerated MLQN method through computer simulations on function approximation and classification problems.

A structure is proposed for high-speed goal recognition by SSD MobileNet using the Coral USB Accelerator. It was confirmed that the goal recognition rate from a long distance is equal to or better than that of conventional methods. Using this method, two CanSat contests were won. The aim of the CanSat competitions is to guide the CanSat to a distance of 0 m from the goal. It is initially guided by GPS but must eventually employ an image recognition model to identify the nearby goal. Zero-meter guidance to the goal was achieved in the Tanegashima Rocket Contest 2018 using a method that recognized the color of the goal by the color of the image, but it was vulnerable to changes in lighting conditions, such as weather changes. Therefore, a deep-learning method for CanSat goal recognition was applied for the first time at ARLISS 2019, and the zero-distance goal was achieved, winning the competition. However, it took more than 10 s for recognition owing to the CPU calculations, making it time consuming to reach the goal. The conventional method uses image classification to recognize the location of a goal by preparing multiple regions of interest (ROIs) in the image and repeating the recognition operations for each ROI. However, this method has a complex algorithm and requires the recognition of more ROIs to recognize goals over long distances, which is computationally time consuming. Object detection is an effective method to identify the location of the target object in an image. However, even if the lightest SSD MobileNet V1 and V2 and a hardware accelerator are used, the computation time may not be short enough because the computer is a Raspberry Pi Zero, the weakest class of Linux computers. In addition, if SSD MobileNet V1 and V2 do not have a sufficiently high recognition rate at long distances from the goal compared with conventional methods, it will be difficult to adapt them to a CanSat. To clarify this, SSD MobileNet V1 and V2 were applied to a Raspberry Pi Zero connected to a Coral USB Accelerator, and the recognition rate and recognition time were investigated at long distances from the goal. It was found that the recognition rate was equivalent to or better than that of the conventional method, even at long distances from the goal, and that the recognition time was sufficiently short (approximately 0.2 s). The effectiveness of the proposed method was evaluated at the Noshiro Space Event 2021 and Asagiri CanSat Drop Test (ACTS) 2021, and the 0-m goal was achieved at both events.

First-order methods such as SGD and Adam are popularly used in training Neural networks. On the other hand, second-order methods have shown to have better performance and faster convergence despite their high computational cost by incorporating the curvature information. While second-order methods determine the step size by line search approaches, first-order methods achieve efficient learning by devising a way to adjust the step size. In this paper, we propose a new learning algorithm for training neural networks by combining first-order and second-order methods. We investigate the effectiveness of our proposed method when combined with popular first-order methods -SGD, Adagrad, and Adam, through experiments using image classification problems.

Incorporating curvature information in stochastic methods has been a challenging task. This paper proposes a momentum accelerated BFGS quasi-Newton method in both its full and limited memory forms, for solving stochastic large scale non-convex optimization problems in neural networks (NN).

This study focuses on the Nesterov's accelerated quasi-Newton (NAQ) method in the context of deep neural networks (DNN) and its applications. The thesis objective is to confirm the robustness and efficiency of Nesterov's acceleration to quasi-Netwon (QN) methods by developing practical algorithms for different fields of optimization problems.