Recursive parameter estimation: asymptotic expansion

Sharia, Teo

doi:10.1007/s10463-008-0179-z

Cited by 8 publications

(11 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Asymptotically under suitable regularity conditions, the mean of our gaussian is guaranteed to converge to the true θ . Consistency can be established by applying theorems for the consistency of estimators based on stochastic gradient descent (Fabian, 1978;Sharia, 2007). We used numerical simulations (data not shown) to verify the predictions of these theorems.…”

Section: Representing and Updating The Posteriormentioning

confidence: 99%

Sequential Optimal Design of Neurophysiology Experiments

2009

View full text Add to dashboard Cite

Adaptively optimizing experiments has the potential to significantly reduce the number of trials needed to build parametric statistical models of neural systems. However, application of adaptive methods to neurophysiology has been limited by severe computational challenges. Since most neurons are high-dimensional systems, optimizing neurophysiology experiments requires computing high-dimensional integrations and optimizations in real time. Here we present a fast algorithm for choosing the most informative stimulus by maximizing the mutual information between the data and the unknown parameters of a generalized linear model (GLM) that we want to fit to the neuron's activity. We rely on important log concavity and asymptotic normality properties of the posterior to facilitate the required computations. Our algorithm requires only low-rank matrix manipulations and a two-dimensional search to choose the optimal stimulus. The average running time of these operations scales quadratically with the dimensionality of the GLM, making real-time adaptive experimental design feasible even for high-dimensional stimulus and parameter spaces. For example, we require roughly 10 milliseconds on a desktop computer to optimize a 100-dimensional stimulus. Despite using some approximations to make the algorithm efficient, our algorithm asymptotically decreases the uncertainty about the model parameters at a rate equal to the maximum rate predicted by an asymptotic analysis. Simulation results show that picking stimuli by maximizing the mutual information can speed up convergence to the optimal values of the parameters by an order of magnitude compared to using random (nonadaptive) stimuli. Finally, applying our design procedure to real neurophysiology experiments requires addressing the nonstationarities that we would expect to see in neural responses; our algorithm can efficiently handle both fast adaptation due to spike history effects and slow, nonsystematic drifts in a neuron's activity.

show abstract

Section: Representing and Updating The Posteriormentioning

confidence: 99%

Sequential Optimal Design of Neurophysiology Experiments

2009

View full text Add to dashboard Cite

show abstract

“…conditions that guarantee a property of type (2.12) for any u, and also conditions on the growth of the corresponding functions at infinity (see [20] for details). Once the convergence is secured, the rate of convergence and the asymptotic distribution depend on the local behaviour of the corresponding functions (like differentiability of higher order) and the ergodicity of the model (see [20]- [22]). The statistical model described in [20]- [22] are quite general with no specific requirements on the dependence structure and the distribution of the underline process.…”

Section: Stochastic Approximation Type Estimation Algorithmsmentioning

confidence: 99%

“…Once the convergence is secured, the rate of convergence and the asymptotic distribution depend on the local behaviour of the corresponding functions (like differentiability of higher order) and the ergodicity of the model (see [20]- [22]). The statistical model described in [20]- [22] are quite general with no specific requirements on the dependence structure and the distribution of the underline process. The conditions in these works are given in terms of the conditional distributions.…”

Section: Stochastic Approximation Type Estimation Algorithmsmentioning

confidence: 99%

Efficient on-line estimation of autoregressive parameters

Sharia

2010

Math. Meth. Stat.

View full text Add to dashboard Cite

New procedures for estimating autoregressive parameters in AR(m) models are proposed. The proposed method allows for incorporation of auxiliary information into the estimation process and, under certain regularity conditions are consistent and asymptotically efficient. Also, these procedures are naturally on-line and do not require storing all the data. Theoretical results are presented in the case when m = 1. Two important particular cases are considered in details: linear procedures and likelihood procedures with the LS truncations. A specific example is also presented to briefly discuss some practical aspects of applications of the procedures of this type.

show abstract

“…Asymptotic behaviour of this type of procedures for non i.i.d. models was studied by a number of authors, see e.g., [7], [9], [18], [24] - [27]. Results in [27] show that to obtain an estimator with asymptotically optimal properties, one has to consider a state-dependent matrix-valued random step-size sequence.…”

Section: Introductionmentioning

confidence: 99%

Truncated stochastic approximation with moving bounds: convergence

Sharia

2014

Stat Inference Stoch Process

Self Cite

View full text Add to dashboard Cite

In this paper we propose a wide class of truncated stochastic approximation procedures with moving random bounds. While we believe that the proposed class of procedures will find its way to a wider range of applications, the main motivation is to accommodate applications to parametric statistical estimation theory. Our class of stochastic approximation procedures has three main characteristics: truncations with random moving bounds, a matrix valued random step-size sequence, and dynamically changing random regression function. We establish convergence and consider several examples to illustrate the results.

show abstract

Recursive parameter estimation: asymptotic expansion

Cited by 8 publications

References 21 publications

Sequential Optimal Design of Neurophysiology Experiments

Sequential Optimal Design of Neurophysiology Experiments

Efficient on-line estimation of autoregressive parameters

Truncated stochastic approximation with moving bounds: convergence

Contact Info

Product

Resources

About