In multi-agent networked systems, parameter estimation problems arising in many practical applications are often required to solve Non-Linear Least Squares (NLLS) problems with the usual objective function (i.e., sum of squared residuals). The aim is to estimate a global parameter of interest across the network, such that the discrepancy between the estimation model and the real output of the system is minimized. There are challenges to face when applying the conventional Gauss-Newton method, such as non-cooperation and prosaic learning behavior. In this paper, we propose two Gauss-Newton type fusion estimation algorithms for solving overdetermined NLLS optimization problems arising frequently in multi-agent networked environment. One is the cycle-based Gauss-Newton (CGN) algorithm that is more attractive in performance due to its distributed nature than its peer: the known centralized Gauss-Newton algorithm. On the basis of CGN, we put emphasis on developing a simple but effective learning scheme leveraging an incremental technique, which is distributed on each computing agent over network. Such scheme results in the Incremental Gauss-Newton (IGN) algorithm that achieves a clear increase on convergence rate at the expense of higher computation cost than the CGN algorithm as well as the centralized one by deeper learning over the networking cycle. Both algorithms utilize Gauss-Newton iteration update in a cyclic cooperative manner, which offers the flexibility in exploiting the network topology. We provide the detailed analysis and the sufficient conditions for convergence of proposed IGN algorithm. By applying to target localization in wireless sensor networks, the numerical results confirm our convergence analysis and show that the proposed incremental scheme outperforms the centralized one in term of convergence performance. INDEX TERMS Gauss-Newton method, Incremental learning, Nonlinear least squares, Cyclic routing, Sequential fusion I. INTRODUCTION F USION estimation [1], [2] has recently attracted a lot of attention in the field of multi-agent networked systems, especially as the agents are equipped with more and more powerful communication and computing components. An example in smart home is that the self-directing vacuum can automatically avoid obstacles by communicating with the intelligent devices deployed in a family area. Over the past decades, the second-order learning methods such as Newton's method for solving general convex optimization problems have been largely overlooked because of high computational load, as compared with first-order meth

No abstract

Non-linear least squares problems arise from data fitting have received recently a lot of attention, particularly for the estimates of the model parameters over networked systems. Although the diffusion Gauss-Newton method offers many advantages for solving the non-linear least squares problem in wireless sensor network to estimate target position parameter, there are some key challenges when applying it to practice, including singularity of Gauss-Newton Hessian, selection to constant step sizes and steady state oscillation. These remaining issues lead to obvious performance degradation such as high computational cost, vulnerability to step size change and resulting instability on estimation.In this paper, to eliminate the singularity, we develop a diffusion Levenberg-Marquardt method such that the problem of constant step size is also addressed together. Then, to reach agreement on estimated vector, a consensus implementation is further proposed, thus eliminating the oscillation during steady state. Consequently, the proposed consensus-based diffusion Levenberg-Marquardt method provides a general solution for the non-linear least squares problems with an objective that takes the form of a sum of squared residual terms. By applying to collaborative localization and distributed optimization arise in large scale machine learning, simulation results confirm the effectiveness and wide applicability of proposed method in terms of convergence rate, accuracy and consistency of estimates.

No abstract

No abstract