Sufficient statistics and intrinsic accuracy

Pitman, E. J. G.; Wishart, John

doi:10.1017/s0305004100019307

Cited by 181 publications

(85 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In (1), the averages of the functions multiplying the model parameters are sufficient statistics [9][10][11], which in the case at hand means that inference of the biases h i and the interaction strengths J ij cannot be done better using all the B samples (N B data points), than by observing just m data points). The optimal estimates (in a maximum likelihood sense) are given by ∂ hi log Z = βm…”

Section: Maximum Likelihood and Computabilitymentioning

confidence: 99%

Inverse Ising Inference Using All the Data

2012

View full text Add to dashboard Cite

We show that a method based on logistic regression, using all the data, solves the inverse Ising problem far better than mean-field calculations relying only on sample pairwise correlation functions, while still computationally feasible for hundreds of nodes. The largest improvement in reconstruction occurs for strong interactions. Using two examples, a diluted Sherrington-Kirkpatrick model and a two-dimensional lattice, we also show that interaction topologies can be recovered from few samples with good accuracy and that the use of l1 regularization is beneficial in this process, pushing inference abilities further into low-temperature regimes.

show abstract

Section: Maximum Likelihood and Computabilitymentioning

confidence: 99%

Inverse Ising Inference Using All the Data

2012

View full text Add to dashboard Cite

show abstract

“…4) that has been flourishing in probability and statistics since the 1930s, with many brilliant results and paperse.g. those by Koopman & Pitman (Koopman 1936;Pitman 1936;Darmois 1935), Diaconis & Freedman (Freedman 1962a,b;Diaconis 1977;1988;1992;Diaconis et al 1980a,b,c;1987;1988;1990), Martin-Löf (1974), Lauritzen (1974a,b;1988;1984;, Ressel (1985), Aldous (1981;1982;1985;, Kallenberg (1989;2005), Cifarelli, Regazzini, Fortini, et al (Cifarelli et al 1979;1982;Regazzini 1996;Fortini et al 2000;2002;2012;, to name very few besides those by de Finetti (1930;1937;1938), already known in the quantum literature. See Dawid's review (2013) for a small glimpse.…”

Section: Resultsmentioning

confidence: 89%

Quantum theory within the probability calculus: a there-you-go theorem and partially exchangeable models

Mana¹

2018

Preprint

View full text Add to dashboard Cite

"Ever since the advent of modern quantum mechanics in the late 1920's, the idea has been prevalent that the classical laws of probability cease, in some sense, to be valid in the new theory. [. . .] The primary object of this presentation is to show that the thesis in question is entirely without validity and is the product of a confused view of the laws of probability" (Koopman, 1957). The secondary objects are: to show that quantum inferences are cases of partially exchangeable statistical models with particular prior constraints; to wonder about such constraints; and to plead for a dialogue between quantum theory and the theory of exchangeable models.

show abstract

“…Exponential pdfs play an important role in statistics due to the Pitman-Koopman-Darmois theorem [29,94,117], which states that among distributions whose domain does not vary with the parameter being estimated, a sufficient statistic with bounded dimension as the sample size increases can be found only in exponential families [100]. Furthermore, efficient estimators achieving the CRB exist only when the underlying model is exponential.…”

Section: Estimation Modelmentioning

confidence: 99%

Rethinking Biased Estimation: Improving Maximum Likelihood and the Cramér–Rao Bound

Eldar

2007

FNT in Signal Processing

View full text Add to dashboard Cite

One of the prime goals of statistical estimation theory is the development of performance bounds when estimating parameters of interest in a given model, as well as constructing estimators that achieve these limits. When the parameters to be estimated are deterministic, a popular approach is to bound the mean-squared error (MSE) achievable within the class of unbiased estimators. Although it is well-known that lower MSE can be obtained by allowing for a bias, in applications it is typically unclear how to choose an appropriate bias.In this survey we introduce MSE bounds that are lower than the unbiased Cramér-Rao bound (CRB) for all values of the unknowns. We then present a general framework for constructing biased estimators with smaller MSE than the standard maximum-likelihood (ML) approach, regardless of the true unknown values. Specializing the results to the linear Gaussian model, we derive a class of estimators that dominate least-squares in terms of MSE. We also introduce methods for choosing regularization parameters in penalized ML estimators that outperform standard techniques such as cross validation.

show abstract

Sufficient statistics and intrinsic accuracy

Cited by 181 publications

References 3 publications

Inverse Ising Inference Using All the Data

Inverse Ising Inference Using All the Data

Quantum theory within the probability calculus: a there-you-go theorem and partially exchangeable models

Rethinking Biased Estimation: Improving Maximum Likelihood and the Cramér–Rao Bound

Contact Info

Product

Resources

About