The impact of rater training on clinical outcomes assessment data: a literature review

Sadler, Michael; Yamamoto, Rinah T.; Khurana, L; Dallabrida, Susan M.

doi:10.18203/2349-3259.ijct20173133

Cited by 17 publications

(13 citation statements)

References 34 publications

(45 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This fits well with other studies where scales and estimates are used. Sadler et al [36] emphasise that lack of education, skills and good assessment routines are more common causes of differences in assessments than the deficiencies in the instruments themselves.…”

Section: Discussion Of Resultsmentioning

confidence: 99%

Factors That Promote Remission in Individuals with Schizophrenia

Olsson-Tall¹,

Marklund²,

Kylén³

et al. 2019

OJN

View full text Add to dashboard Cite

The knowledge about factors contributing to remission among individuals with schizophrenia is still missing. Studies show that the number of individuals' reaching remission is varying among psychiatric outpatient care centres. Therefore, the purpose was to investigate case managers (CM) perception of care interventions promoting remission. Semi-structured interviews were conducted with CM (n = 12) and data were analysed with qualitative content analysis. The results show that CM needs to possess competence, master assessment structure and process. These key-points appear to be of crucial importance for a successful promotion of remission among individuals with schizophrenia.

show abstract

Section: Discussion Of Resultsmentioning

confidence: 99%

Factors That Promote Remission in Individuals with Schizophrenia

Olsson-Tall¹,

Marklund²,

Kylén³

et al. 2019

OJN

View full text Add to dashboard Cite

show abstract

“…This may mitigate some of the relatively small differences in the scores; however, it will be important to evaluate the performance of these scoring methods in larger studies and the impact on responsiveness of the instrument. Lastly, inter‐rater reliability can be improved with rater training, such as an online didactic and scored practice session . Studies of psoriasis outcome instruments showed significant improvement in reliability after training …”

Section: Discussionmentioning

confidence: 99%

“…Lastly, inter-rater reliability can be improved with rater training, such as an online didactic and scored practice session. 48 Studies of psoriasis outcome instruments showed significant improvement in reliability after training. 49,50 The intrarater, or test-retest, reliability of the SASH was very high at 0Á98 (95% CI 0Á94-1Á00) and in this study was higher than all other outcome instruments.…”

Section: Discussionmentioning

confidence: 99%

Severity and Area Score for Hidradenitis ( SASH ): a novel outcome measurement for hidradenitis suppurativa

2019

View full text Add to dashboard Cite

Summary Background Hidradenitis suppurativa (HS) is a chronic, inflammatory skin disease with a large impact on patients’ health‐related quality of life. However, reliable and consistent outcome measures to assess body surface area (BSA) of HS have not been established. Objectives To develop and assess the reliability and validity of a novel outcome instrument for assessment of HS BSA in a clinical trial setting. Methods Qualitative interviews and focus groups were conducted from July to August 2015 and October 2017 to January 2018. Evaluation of the measurement was assessed during a single‐day grading session with patients in April 2018. Participants, who included clinicians or patients, were recruited from academic medical centres in the U.S. mid‐Atlantic region. Results Concept elicitation included input from 10 providers, of which 60% (n = 6) were female, 80% (n = 8) dermatology specialists and 20% (n = 2) gynaecology specialists. Cognitive debriefing was conducted with 11 providers, of which 82% (n = 9) were dermatologists and 18% (n = 2) gynaecologists. The evaluation stage included 10 clinicians and 23 patients. The intraclass correlation coefficient (ICC) for inter‐rater reliability was 0·60 [95% confidence interval (CI) 0·44–0·74]. The ICC for intrarater reliability was 0·98 (95% CI 0·94–1·00). Transformation of the BSA score resulted in an increase in inter‐rater reliability to 0·75 (95% CI 0·62–0·85) or 0·76 (95% CI 0·62–0·85). Scores all demonstrated concurrent validity, with statistically significant correlations with extant scoring methods. Conclusions This novel scale is a reliable and valid HS outcome instrument and may capture a wide range of patients by assessing BSA. Future research is necessary to demonstrate its responsiveness. What's already known about this topic? The major HS disease activity scales rely on lesions counts and have moderate‐to‐good reliability. Body surface area (BSA) is one of the physical signs included in the Core Outcome Set for HS, but is not a part of existing HS disease activity scales. What does this study add? A novel disease severity scale, the Severity and Area Score for Hidradenitis (SASH), was developed and the psychometric properties assessed. There was high inter‐rater reliability of 0·75 and 0·76 when BSA was scored on an ordinal scale, and an excellent intrarater reliability of 0·98. The SASH score also demonstrated convergent validity with extant instruments. What are the clinical implications of this work? The ability of clinicians to accurately assess disease status will be improved. Implementation of the SASH score will help guide and assess the effectiveness of appropriate treatment choice.

show abstract

“…Subjective assessment is an unavoidable element of the screening process for many medical conditions such as psoriasis (Gonin, Lipsitz, Fitzmaurice, & Molenberghs, 2000), Alzheimer's disease (Chaves et al., 2007), and cancer (Ciatto et al., 2006; Elmore et al., 2015; Gard, Aiello Bowles, Miglioretti, Taplin, & Rutter, 2015; Kerlikowske et al., 1998), often leading to excessive variability in clinical outcome measurements (Sadler, Yamamoto, Khurana, & Dallabrida, 2017). Inconsistencies are reported not only between different experts' ratings (interrater agreement) but also between an expert's own repeated ratings (reproducibility or intrarater agreement) of patients' test results.…”

Section: Introductionmentioning

confidence: 99%

“…Poor intrarater and interrater agreement between raters in clinical settings can lead to added complexity when making clinical decisions and has further implications for patient care. In efforts to improve raters' interpretive performance, studies are often conducted in clinical settings to investigate the effectiveness of a new training tool, such as teaching, seminars, online, and practical instruction (Antonio & Crispi, 2010; Grassano, Pagana, Daperno, Bibbona, & Gasparini, 2018; Sadler et al., 2017).…”

Section: Introductionmentioning

confidence: 99%