Low Power Sparse Approximation on Reconfigurable Analog Hardware

Shapero, Samuel; Charles, Adam S.; Rozell, C.; Hasler, P.

doi:10.1109/jetcas.2012.2214615

Cited by 32 publications

(34 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The network architecture being used in this study provably solves the optimization in Eq. (2) with strong convergence guarantees [27], can implement many variations of the sparse coding hypothesis (i.e., different sparsity-inducing cost functions) [28], and is implementable in neuromorphic analog circuits [29].…”

Section: Resultsmentioning

confidence: 99%

Visual Nonclassical Receptive Field Effects Emerge from Sparse Coding in a Dynamical System

2013

Self Cite

View full text Add to dashboard Cite

Extensive electrophysiology studies have shown that many V1 simple cells have nonlinear response properties to stimuli within their classical receptive field (CRF) and receive contextual influence from stimuli outside the CRF modulating the cell's response. Models seeking to explain these non-classical receptive field (nCRF) effects in terms of circuit mechanisms, input-output descriptions, or individual visual tasks provide limited insight into the functional significance of these response properties, because they do not connect the full range of nCRF effects to optimal sensory coding strategies. The (population) sparse coding hypothesis conjectures an optimal sensory coding approach where a neural population uses as few active units as possible to represent a stimulus. We demonstrate that a wide variety of nCRF effects are emergent properties of a single sparse coding model implemented in a neurally plausible network structure (requiring no parameter tuning to produce different effects). Specifically, we replicate a wide variety of nCRF electrophysiology experiments (e.g., end-stopping, surround suppression, contrast invariance of orientation tuning, cross-orientation suppression, etc.) on a dynamical system implementing sparse coding, showing that this model produces individual units that reproduce the canonical nCRF effects. Furthermore, when the population diversity of an nCRF effect has also been reported in the literature, we show that this model produces many of the same population characteristics. These results show that the sparse coding hypothesis, when coupled with a biophysically plausible implementation, can provide a unified high-level functional interpretation to many response properties that have generally been viewed through distinct mechanistic or phenomenological models.

show abstract

Section: Resultsmentioning

confidence: 99%

Visual Nonclassical Receptive Field Effects Emerge from Sparse Coding in a Dynamical System

2013

Self Cite

View full text Add to dashboard Cite

show abstract

“…The design and implementation of analog circuits have traditionally been difficult, but recent advances in reconfigurable analog circuits (Twigg & Hasler, 2009) have improved many of the issues related to the design of these systems. In fact, the re-configurable platform described in Twigg and Hasler has been used to implement a small version of the LCA for solving BPDN (Shapero, Charles et al, in press; Shapero, Rozell et al, 2012), and preliminary tests of this implementation are consistent with simulations of the idealized LCA. These results lend encouragement to the idea that efficient analog circuits could be implemented for the variety of cost functions described in this letter.…”

Section: Discussionmentioning

confidence: 97%

“…In particular, each node consists of a leaky integrator and a nonlinear thresholding function, and it is driven by both feed-forward and lateral (inhibitory and excitatory) recurrent connections. This architecture has been implemented in neuromorphic hardware as a purely analog system (Shapero, Charles et al, in press) and by using integrate-and-fire spiking neurons for each node (Shapero, Rozell, et al, 2012). We also note that other types of network structures have also been proposed recently to approximately solve specific versions of the sparse approximation problem (Rehn & Sommer, 2007; Perrinet et al, 2004; Zylberberg et al, 2011; Hu, Genkin, & Chklovskii, 2012).…”

Section: Background and Related Workmentioning

confidence: 99%

“…Interestingly, the sparse coding problem has become very prominent in modern signal processing, for example, for use in inverse problems (Elad, Figueiredo, & Ma, 2010) and computer vision (Wright et al, 2010). There is also increasing interest in leveraging the computational benefits of analog neuromorphic architectures for these problems (Shapero, Charles, Rozell, & Hasler, in press; Shapero, Rozell, & Hasler, 2012). …”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Common Network Architecture Efficiently Implements a Variety of Sparsity-Based Inference Problems

Charles

Garrigues²,

Rozell

2012

Neural Computation

Self Cite

View full text Add to dashboard Cite

The sparse coding hypothesis has generated significant interest in the computational and theoretical neuroscience communities, but there remain open questions about the exact quantitative form of the sparsity penalty and the implementation of such a coding rule in neurally plausible architectures. The main contribution of this work is to show that a wide variety of sparsity-based probabilistic inference problems proposed in the signal processing and statistics literatures can be implemented exactly in the common network architecture known as the locally competitive algorithm (LCA). Among the cost functions we examine are approximate ℓp norms (0 ≤ p ≤ 2), modified ℓp-norms, block-ℓ1 norms, and reweighted algorithms. Of particular interest is that we show significantly increased performance in reweighted ℓ1 algorithms by inferring all parameters jointly in a dynamical system rather than using an iterative approach native to digital computational architectures.

show abstract

“…where σ 2 is the measurement noise variance and the signal estimate is x = W z. BPDN has been a particularly popular approach due to its strong performance guarantees [7] and the development of many specialized optimization approaches [8]- [13]. As a guarantee of the measurement quality, we say that Φ satisfies the restricted isometry property with parameters 2S and δ (RIP(2S,δ)) with respect to W if for every 2S-sparse vector z we have that…”

Section: A Sparse Signal Estimationmentioning

confidence: 99%

Dynamic Filtering of Time-Varying Sparse Signals via <inline-formula> <tex-math notation="LaTeX">$\ell _1$</tex-math> </inline-formula> Minimization

Charles

Balavoine

Rozell

2016

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

Despite the importance of sparsity signal models and the increasing prevalence of high-dimensional streaming data, there are relatively few algorithms for dynamic filtering of time-varying sparse signals. Of the existing algorithms, fewer still provide strong performance guarantees. This paper examines two algorithms for dynamic filtering of sparse signals that are based on efficient 1 optimization methods. We first present an analysis for one simple algorithm (BPDN-DF) that works well when the system dynamics are known exactly. We then introduce a novel second algorithm (RWL1-DF) that is more computationally complex than BPDN-DF but performs better in practice, especially in the case where the system dynamics model is inaccurate. Robustness to model inaccuracy is achieved by using a hierarchical probabilistic data model and propagating higher-order statistics from the previous estimate (akin to Kalman filtering) in the sparse inference process. We demonstrate the properties of these algorithms on both simulated data as well as natural video sequences. Taken together, the algorithms presented in this paper represent the first strong performance analysis of dynamic filtering algorithms for time-varying sparse signals as well as state-of-the-art performance in this emerging application.

show abstract

Low Power Sparse Approximation on Reconfigurable Analog Hardware

Cited by 32 publications

References 29 publications

Visual Nonclassical Receptive Field Effects Emerge from Sparse Coding in a Dynamical System

Visual Nonclassical Receptive Field Effects Emerge from Sparse Coding in a Dynamical System

A Common Network Architecture Efficiently Implements a Variety of Sparsity-Based Inference Problems

Dynamic Filtering of Time-Varying Sparse Signals via <inline-formula> <tex-math notation="LaTeX">$\ell _1$</tex-math> </inline-formula> Minimization

Contact Info

Product

Resources

About