We present an analysis of the Locally Competitive Algotihm (LCA), which is a Hopfield-style neural network that efficiently solves sparse approximation problems (e.g., approximating a vector from a dictionary using just a few nonzero coefficients). This class of problems plays a significant role in both theories of neural coding and applications in signal processing. However, the LCA lacks analysis of its convergence properties, and previous results on neural networks for nonsmooth optimization do not apply to the specifics of the LCA architecture. We show that the LCA has desirable convergence properties, such as stability and global convergence to the optimum of the objective function when it is unique. Under some mild conditions, the support of the solution is also proven to be reached in finite time. Furthermore, some restrictions on the problem specifics allow us to characterize the convergence rate of the system by showing that the LCA converges exponentially fast with an analytically bounded convergence rate. We support our analysis with several illustrative simulations.

There exist many well-established techniques to recover sparse signals from compressed measurements with known performance guarantees in the static case. More recently, new methods have been proposed to tackle the recovery of time-varying signals, but few benefit from a theoretical analysis. In this paper, we give theoretical guarantees for the Iterative Soft-Thresholding Algorithm (ISTA) and its continuous-time analogue the Locally Competitive Algorithm (LCA) to perform this tracking in real time. ISTA is a well-known digital solver for static sparse recovery, whose iteration is a first-order discretization of the LCA differential equation. Our analysis is based on the Restricted Isometry Property (RIP) and shows that the outputs of both algorithms can track a time-varying signal while compressed measurements are streaming, even when no convergence criterion is imposed at each time step. The 2-distance between the target signal and the outputs of both discrete-and continuous-time solvers is shown to decay to a bound that is essentially optimal. Our analysis is supported by simulations on both synthetic and real data.In addition, the form of the activation function (3) Using (9) andhypothesis (16), we getησ 1053-587X (c) 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TSP.2015.2420535, IEEE Transactions on Signal Processing 13 ≤ λ √ q.Since ∆[l + 1] ⊂ J[l + 1], the induction hypothesis (25) holds at l + 1. As a consequence, we proved 1) of the theorem and also the stronger result u J[l] [l] 2 ≤ λ √ q, ∀l ≥ 1, which will be used in the following.Next, we show by induction on l that (17) holds ∀l ≥ 0. It obviously holds for l = 0. Next, assume that for some l ≥ 0, (17) holds. There exist a unique k ≥ 0 and a unique 0 ≤ i ≤ P − 1 such that l = kP + i. In the previous part of the proof, we showed that u J [l + 2] 2 ≤ λ √ q, where J = J[l + 2] = ∆[l + 2] ∪ Γ[l + 1] ∪ Γ † [l + 1] and that J contains less than S + 2q indices. As a consequence, we can use the RIP of Φ T J Φ J and get a[l + 2] − a † [l + 1] 2 ≤ a[l + 2] − u J [l + 2] 2 + u J [l + 2] − a † [l + 1] 2 ≤ u J [l + 2] 2 + u J [l + 2] − a † [l + 1] 2 ≤ λ √ q + ηΦ T J [l + 1] + ηΦ T J Φ J − I J a † [l + 1] − a[l + 1] 2 ≤ λ √ q + ησ + c a † [l + 1] − a[l + 1] 2

Abstract-This paper studies the convergence rate of a continuous-time dynamical system for 1-minimization, known as the Locally Competitive Algorithm (LCA). Solving 1-minimization problems efficiently and rapidly is of great interest to the signal processing community, as these programs have been shown to recover sparse solutions to underdetermined systems of linear equations and come with strong performance guarantees. The LCA under study differs from the typical 1-solver in that it operates in continuous time: instead of being specified by discrete iterations, it evolves according to a system of nonlinear ordinary differential equations. The LCA is constructed from simple components, giving it the potential to be implemented as a large-scale analog circuit. The goal of this paper is to give guarantees on the convergence time of the LCA system. To do so, we analyze how the LCA evolves as it is recovering a sparse signal from underdetermined measurements. We show that under appropriate conditions on the measurement matrix and the problem parameters, the path the LCA follows can be described as a sequence of linear differential equations, each with a small number of active variables. This allows us to relate the convergence time of the system to the restricted isometry constant of the matrix. Interesting parallels to sparse-recovery digital solvers emerge from this study. Our analysis covers both the noisy and noiseless settings and is supported by simulation results.

Analog circuits and systems research and education can benefit from the flexibility provided by large-scale Field Programmable Analog Arrays (FPAAs). This paper presents the hardware and software infrastructure supporting the use of a family of floating-gate based FPAAs being developed at Georgia Tech. This infrastructure is compact and portable and provides the user with a comprehensive set of tools for custom analog circuit design and implementation. The infrastructure includes the FPAA IC, discrete ADC, DAC and amplifier ICs, a 32-Bit ARM based microcontroller for interfacing the FPAA with the user's computer, and Matlab and targeting software. The FPAA hardware communicates with Matlab over a USB connection. The USB connection also provides the hardware's power. The software tools include three major systems: a Matlab Simulink FPAA program, a SPICE to FPAA compiler called GRASPER, and a visualization tool called RAT. The hardware consists of two custom PCB designs which include a main board used to program and control an FPAA IC and an FPAA IC adaptor board used to interface a QFP packaged FPAA IC with the 100 pin ZIF socket on the main programming and control board.

No abstract