1979
DOI: 10.2307/2346806
|View full text |Cite
|
Sign up to set email alerts
|

Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm

Abstract: Summary In compiling a patient record many facets are subject to errors of measurement. A model is presented which allows individual error‐rates to be estimated for polytomous facets even when the patient's “true” response is not available. The EM algorithm is shown to provide a slow but sure way of obtaining maximum likelihood estimates of the parameters of interest. Some preliminary experience is reported and the limitations of the method are described.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1
1
1

Citation Types

4
1,174
0
2

Year Published

1997
1997
2017
2017

Publication Types

Select...
4
3
1

Relationship

0
8

Authors

Journals

citations
Cited by 1,166 publications
(1,180 citation statements)
references
References 7 publications
4
1,174
0
2
Order By: Relevance
“…For errors-in-variables models, this problem has been discussed by Oberski and Satorra (2013). For misclassification models, we suggest looking into Bayesian models based on Dawid and Skene (1979) and Passonneau and Carpenter (2014). Ultimately, a full Bayesian model using all the data to simultaneously estimate and correct for measurement error seems the most promising approach to the problem, although it might be a while until an out-of-the-box solution is available to communication scholars.…”
Section: Future Methodological Researchmentioning
confidence: 99%
“…For errors-in-variables models, this problem has been discussed by Oberski and Satorra (2013). For misclassification models, we suggest looking into Bayesian models based on Dawid and Skene (1979) and Passonneau and Carpenter (2014). Ultimately, a full Bayesian model using all the data to simultaneously estimate and correct for measurement error seems the most promising approach to the problem, although it might be a while until an out-of-the-box solution is available to communication scholars.…”
Section: Future Methodological Researchmentioning
confidence: 99%
“…A related important problem is how in practice to assess the generalization performance of a learned model with uncertain labels (Smyth et al, 1994b), which we do not consider in this paper. Prior research has addressed important problems necessary for a full labeling solution that uses multiple noisy labelers, such as estimating the quality of labelers (Dawid and Skene, 1979;Donmez et al, 2009Donmez et al, , 2010Smyth, 1996;Smyth et al, 1994b), and learning with uncertain labels (Lugosi, 1992;Silverman, 1980;Smyth, 1995). Raykar et al (2009Raykar et al ( , 2010) recently presented a model that built on and expanded this line of work, and showed how to integrate the process of concurrently building classifier and learning the quality of the labelers.…”
Section: Related Workmentioning
confidence: 99%
“…We sidestep the issue of knowing p j : the techniques we present do not rely on this knowledge and are largely agnostic about the quality of the labelers. Inferring p j accurately should lead to improved techniques; Dawid and Skene (1979) and Smyth et al (Smyth, 1996;Smyth et al, 1994b) have shown how to use an expectation-maximization framework for estimating the quality of labelers. We also assume for simplicity that each labeler j only gives one label, but that is not a restrictive assumption in what follows.…”
Section: Notation and Assumptionsmentioning
confidence: 99%
See 1 more Smart Citation
“…A second example is automatic object discovery in images (for example [15]). [7] proposed an expectation maximisation procedure to calculate a posterior distribution over some true latent class, conditioned on the subjective labeling, a technique that has been used more recently by [15]. As a pre-processing step, this seems more sensible than taking the majority but it relies completely on the label information and does not use covariate information.…”
Section: Introductionmentioning
confidence: 99%