An assessment of the inter-rater reliability of the ASA physical status score in the orthopaedic trauma population

Ihejirika-Lomedico, Rivka; Thakore, Rachel V.; Sathiyakumar, Vasanth; Ehrenfeld, Jesse M.; Obremskey, William T.; Sethi, Manish K.

doi:10.1016/j.injury.2014.02.039

Cited by 39 publications

(39 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…An interpretation of Fleiss’ kappa has not yet been established because the number of categories and subjects affects the magnitude of the value. Ihejirika and colleagues analyzed the inter‐rater reliability of ASA‐PS using nine scenarios, similar to the present study, and obtained moderate agreement (κ = 0.51). They concluded that substantial agreement strength for reliability was achieved .…”

Section: Discussionsupporting

confidence: 75%

“…Ihejirika and colleagues analyzed the inter‐rater reliability of ASA‐PS using nine scenarios, similar to the present study, and obtained moderate agreement (κ = 0.51). They concluded that substantial agreement strength for reliability was achieved . The present study achieved a higher value for F‐ κ (0.55) than that reported by Ihejirika and colleagues .…”

Section: Discussionsupporting

confidence: 75%

“…They concluded that substantial agreement strength for reliability was achieved . The present study achieved a higher value for F‐ κ (0.55) than that reported by Ihejirika and colleagues . The ICC is a measure of inter‐rater reliability for quantitative ratings.…”

Section: Discussionsupporting

confidence: 50%

“…Sample size was determined as 50 based on the assumption that ICC was 0.7 with a half width of 95% confidence interval of 0.2. We also analyzed the agreement between ASA‐PS ratings using Fleiss’ kappa . These indices ranged between 0 and 1.0, with a higher value indicating greater consistency.…”

Section: Methodsmentioning

confidence: 99%

See 3 more Smart Citations

Inter‐rater reliability of the American Society of Anesthesiologists physical status rating for emergency gastrointestinal surgery

Shichino

Hirao

Haga

2016

Acute Medicine & Surgery

View full text Add to dashboard Cite

Aim: The American Society of Anesthesiologists Physical Status (ASA-PS) classification system is used worldwide and has also been incorporated into various prediction rules. However, concerns have been raised regarding inter-rater agreement in various surgical fields. Although emergency gastrointestinal surgery is relatively common and associated with high postoperative mortality, a reliability study has not yet been undertaken in this field. The aim of the present study was to investigate the inter-rater reliability of ASA-PS for emergency gastrointestinal surgery.Methods: Three sets of scenarios were generated for each ASA-PS class (2E, 3E, and 4E) in emergency gastrointestinal surgery, resulting in nine scenarios. These scenarios described the preoperative profiles of patients in one hospital. Two or three anesthesiologists from 18 other hospitals provided scores for ASA-PS for each scenario.Results: Fifty anesthesiologists scored the ASA-PS class. Between 66% and 90% of these anesthesiologists assigned the same ratings as the reference ratings for the individual scenarios. Inter-rater reliability was assessed using Fleiss' kappa (95% confidence interval) of 0.55 (0.54-0.56, P < 0.001) and an intraclass correlation coefficient (95% confidence interval) of 0.79 (0.63-0.93, P < 0.001). Conclusion:The results of the present study revealed the consistency of ASA-PS ratings between anesthesiologists for emergency gastrointestinal surgery. The ASA-PS may serve as a reliable variable in the prediction rules for this field.

show abstract

Section: Discussionsupporting

confidence: 75%

Section: Discussionsupporting

confidence: 75%

Section: Discussionsupporting

confidence: 50%

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Inter‐rater reliability of the American Society of Anesthesiologists physical status rating for emergency gastrointestinal surgery

Shichino

Hirao

Haga

2016

Acute Medicine & Surgery

View full text Add to dashboard Cite

show abstract

“…The decrease in the kappa value associated with increasing the level of subclassification is consistent with results from other nested classification systems in the orthopaedic literature. 19-21 When these independent assessments were pooled in the final consensus meeting, it was noted that critical pieces of information, such as cultures or laboratory results that would be required for classification under the CDC system, were either not recorded or not routinely collected in standard patient follow-up. As such, the results of the study do not support using the full CDC classification system to differentiate between levels of post-operative infection when chart data are reviewed retrospectively.…”

Section: Discussionmentioning

confidence: 99%

The inter-rater reliability of the diagnosis of surgical site infection in the context of a clinical trial

Nuttall

Evaniew

Thornley

et al. 2016

Bone & Joint Research

View full text Add to dashboard Cite

ObjectivesThe diagnosis of surgical site infection following endoprosthetic reconstruction for bone tumours is frequently a subjective diagnosis. Large clinical trials use blinded Central Adjudication Committees (CACs) to minimise the variability and bias associated with assessing a clinical outcome. The aim of this study was to determine the level of inter-rater and intra-rater agreement in the diagnosis of surgical site infection in the context of a clinical trial.Materials and MethodsThe Prophylactic Antibiotic Regimens in Tumour Surgery (PARITY) trial CAC adjudicated 29 non-PARITY cases of lower extremity endoprosthetic reconstruction. The CAC members classified each case according to the Centers for Disease Control (CDC) criteria for surgical site infection (superficial, deep, or organ space). Combinatorial analysis was used to calculate the smallest CAC panel size required to maximise agreement. A final meeting was held to establish a consensus.ResultsFull or near consensus was reached in 20 of the 29 cases. The Fleiss kappa value was calculated as 0.44 (95% confidence interval (CI) 0.35 to 0.53), or moderate agreement. The greatest statistical agreement was observed in the outcome of no infection, 0.61 (95% CI 0.49 to 0.72, substantial agreement). Panelists reached a full consensus in 12 of 29 cases and near consensus in five of 29 cases when CDC criteria were used (superficial, deep or organ space). A stable maximum Fleiss kappa of 0.46 (95% CI 0.50 to 0.35) at CAC sizes greater than three members was obtained.ConclusionsThere is substantial agreement among the members of the PARITY CAC regarding the presence or absence of surgical site infection. Agreement on the level of infection, however, is more challenging. Additional clinical information routinely collected by the prospective PARITY trial may improve the discriminatory capacity of the CAC in the parent study for the diagnosis of infection.Cite this article: J. Nuttall, N. Evaniew, P. Thornley, A. Griffin, B. Deheshi, T. O’Shea, J. Wunder, P. Ferguson, R. L. Randall, R. Turcotte, P. Schneider, P. McKay, M. Bhandari, M. Ghert. The inter-rater reliability of the diagnosis of surgical site infection in the context of a clinical trial. Bone Joint Res 2016;5:347–352. DOI: 10.1302/2046-3758.58.BJR-2016-0036.R1.

show abstract

mTICCS and its inter-rater reliability to predict the need for massive transfusion in severely injured patients

Horst

Lichte

Bläsius

et al. 2020

Eur J Trauma Emerg Surg

View full text Add to dashboard Cite

Purpose The modified Trauma-Induced Coagulopathy Clinical Score (mTICCS) presents a new scoring system for the early detection of the need for a massive transfusion (MT). This easily applicable score was validated in a large trauma cohort and proven comparable to more established complex scoring systems. However, the inter-rater reliability of the mTICCS has not yet been investigated. Methods Therefore, a dataset of 15 randomly selected and severely injured patients (ISS ≥ 16) derived from the database of a level I trauma centre (2010–2015) was used. Moreover, 15 severely injured subjects that received MT were chosen from the same databank. A web-based survey was sent to medical professionals working in the field of trauma care asking them to evaluate each patient using the mTICCS. Results In total, 16 raters (9 residents and 7 specialists) completed the survey. Ratings from 15 medical professionals could be evaluated and led to an ICC of 0.7587 (95% Bootstrap confidence interval (BCI) 0.7149–0.8283). A comparison of working experience specific ICC (n = 7 specialists, ICC: 0.7558, BCI: 0.7076–0.8270; n = 8 residents, ICC: 0.7634, BCI: 0.7183–0.8335) showed no significant difference between the two groups (p = 0.67). Conclusion In summary, reliability values need to be considered when making clinical decisions based on scoring systems. Due to its easy applicability and its almost perfect inter-rater reliability, even with non-specialists, the mTICCS might therefore be a useful tool to predict the early need for MT in multiple trauma.

show abstract

An assessment of the inter-rater reliability of the ASA physical status score in the orthopaedic trauma population

Cited by 39 publications

References 22 publications

Inter‐rater reliability of the American Society of Anesthesiologists physical status rating for emergency gastrointestinal surgery

Inter‐rater reliability of the American Society of Anesthesiologists physical status rating for emergency gastrointestinal surgery

The inter-rater reliability of the diagnosis of surgical site infection in the context of a clinical trial

mTICCS and its inter-rater reliability to predict the need for massive transfusion in severely injured patients

Contact Info

Product

Resources

About