The Measurement of Observer Disagreement in the Recording of Signs

Armitage, P.; Blendis, Laurence M.; Smyllie, H. C.

doi:10.2307/2343899

Cited by 50 publications

(16 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CALCULATION OF RESULTS There are several possible ways of calculating observer error (Armitage, Blendis, and Smyllie, 1966). We have chosen to use the standard deviation agreement index (S.D.A.I.…”

Section: Methodsmentioning

confidence: 99%

Repeatability of physical signs in airways obstruction

Godfrey¹,

Edwards²,

Campbell³

et al. 1969

Thorax

Self Cite

View full text Add to dashboard Cite

We have examined the observer variation in the detection of physical signs in airways obstruction. In the first study, 10 relatively experienced physicians examined 11 patients for the presence of six familiar signs and seven less well-known signs. Six experienced observers then underwent a training period after which they examined another group of 21 patients. The repeatability of all the signs fell about midway between that expected by chance and the maximum possible. There was no difference between the familiar and unfamiliar signs. The training period resulted in a slight but not significant improvement in reliability.We have examined the observer error in the elicitation of some familiar and some less well-known physical signs in patients with airways obstruction. To provide additional information on the importance of skill and training, the study was undertaken in two stages. In the initial study a group of physicians were briefly instructed in the elicitation of the signs, some of which were relatively unfamiliar, and they then examined a series of patients. In the second stage, a smaller group of the more experienced members of the team were trained together for a time and they then examined another series of patients. METHODSIn the initial study, 10 observers examined 11 patients. These observers were all physicians at the Hammersmith Hospital. Seven were members of the Royal College of Physicians, one was of equivalent American standard, and two were post-registration house physicians. In the second study, six observers examined 21 patients. These observers were all members of the Royal College of Physicians and ranged in experience from consultant to registrar. The patients for both studies were selected from the wards and outpatient clinics, and their co-operation was sought after the nature of the study had been explained to them.PHYSICAL SIGNS All patients were examined while reclining on a couch with the back rest raised to 45'. The following list of physical signs is based on tPresent address:

show abstract

Section: Methodsmentioning

confidence: 99%

Repeatability of physical signs in airways obstruction

Godfrey¹,

Edwards²,

Campbell³

et al. 1969

Thorax

Self Cite

View full text Add to dashboard Cite

show abstract

“…• include a "standard deviation agreement" by Armitage lllill., Rogot and Goldberg's "agreement index", and Goodman and Kruskal's lambda (150)(151)(152). F1eiss has shown that, with only minor modifications, they are ail rnathematically equivalent to the kappa coefficient (146).…”

Section: Sample Size Calculationmentioning

confidence: 99%

Evaluation of the Quality of an Injury Surveillance System

Macarthur

Pless

1999

American Journal of Epidemiology

View full text Add to dashboard Cite

“…It takes values between 0 (when there is no agreement at all) and 1 (when the agreement is complete). Several authors use this measure as the starting point and then define their indices applying some transformations to it (Armitage, Blendis, & Smyllie, 1966).…”

Section: Measures Of Agreementmentioning

confidence: 99%

A Monte Carlo–Based Bayesian Approach for Measuring Agreement in a Qualitative Scale

Calle-Alonso

Pérez

2014

Applied Psychological Measurement

View full text Add to dashboard Cite

Agreement analysis has been an active research area whose techniques have been widely applied in psychology and other fields. However, statistical agreement among raters has been mainly considered from a classical statistics point of view. Bayesian methodology is a viable alternative that allows the inclusion of subjective initial information coming from expert opinions, personal judgments, or historical data. A Bayesian approach is proposed by providing a unified Monte Carlo-based framework to estimate all types of measures of agreement in a qualitative scale of response. The approach is conceptually simple and it has a low computational cost. Both informative and non-informative scenarios are considered. In case no initial information is available, the results are in line with the classical methodology, but providing more information on the measures of agreement. For the informative case, some guidelines are presented to elicitate the prior distribution. The approach has been applied to two applications related to schizophrenia diagnosis and sensory analysis.

show abstract

The Measurement of Observer Disagreement in the Recording of Signs

Cited by 50 publications

References 2 publications

Repeatability of physical signs in airways obstruction

Repeatability of physical signs in airways obstruction

Evaluation of the Quality of an Injury Surveillance System

A Monte Carlo–Based Bayesian Approach for Measuring Agreement in a Qualitative Scale

Contact Info

Product

Resources

About