Comparison of Various Metrics for Evaluating the Performance of Deep Learning Binary Classification, Particularly When Underlying Imaging Data Are Imbalanced

Liu, S.; Sun, X.; Roemer, F.W.; Guermazi, A.; Kwoh, C.K.

doi:10.1016/j.ostima.2022.100017

Cited by 3 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior DL work uses MOAKS to determine severity of cartilage damage [55]. Other work predicts other features of MOAKS, bone bruises [68] or inflammation [69]. This is the first quantification of MOAKS osteophyte and cartilage health, demonstrating that NSMs encode this important information that is currently prohibitive to obtain clinically, and costly for research and clinical trials.…”

Section: Methodsmentioning

confidence: 99%

ShapeMed-Knee: A Dataset and Neural Shape Model Benchmark for Modeling 3D Femurs

Gatti,

Blankemeier,

Van Veen

et al. 2024

Preprint

View full text Add to dashboard Cite

Analyzing anatomic shapes of tissues and organs is pivotal for accurate disease diagnostics and clinical decision-making. One prominent disease that depends on anatomic shape analysis is osteoarthritis, which affects 30 million Americans. To advance osteoarthritis diagnostics and prognostics, we introduce ShapeMed-Knee, a 3D shape dataset with 9,376 high-resolution, medical imaging-based 3D shapes of both femur bone and cartilage. Besides data, ShapeMed-Knee includes two benchmarks for assessing reconstruction accuracy and five clinical prediction tasks that assess the utility of learned shape representations. Leveraging ShapeMed-Knee, we develop and evaluate a novel hybrid explicit-implicit neural shape model which achieves up to 40% better reconstruction accuracy than a statistical shape model and implicit neural shape model. Our hybrid models achieve state-of-the-art performance for preserving cartilage biomarkers; they are also the first models to successfully predict localized structural features of osteoarthritis, outperforming shape models and convolutional neural networks applied to raw magnetic resonance images and segmentations. The ShapeMed-Knee dataset provides medical evaluations to reconstruct multiple anatomic surfaces and embed meaningful disease-specific information. ShapeMed-Knee reduces barriers to applying 3D modeling in medicine, and our benchmarks highlight that advancements in 3D modeling can enhance the diagnosis and risk stratification for complex diseases. The dataset, code, and benchmarks will be made freely accessible.

show abstract

Section: Methodsmentioning

confidence: 99%

ShapeMed-Knee: A Dataset and Neural Shape Model Benchmark for Modeling 3D Femurs

Gatti,

Blankemeier,

Van Veen

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…However, recent concerns have arisen regarding the suitability of commonly used performance metrics for AI models, such as AUC, especially when dealing with large datasets characterized by imbalanced data, a frequent scenario in MRI-based structural evaluations of OA-affected joints. 95,96 It is imperative to explore how well OA risk assessment models can be translated to the more heterogeneous MRI datasets typically acquired in clinical practice and clinical trials.…”

Section: Artificial Intelligencementioning

confidence: 99%

Imaging Biomarkers of Osteoarthritis

Roemer,

Wirth,

Demehri

et al. 2024

Semin Musculoskelet Radiol

View full text Add to dashboard Cite

Currently no disease-modifying osteoarthritis drug has been approved for the treatment of osteoarthritis (OA) that can reverse, hold, or slow the progression of structural damage of OA-affected joints. The reasons for failure are manifold and include the heterogeneity of structural disease of the OA joint at trial inclusion, and the sensitivity of biomarkers used to measure a potential treatment effect.This article discusses the role and potential of different imaging biomarkers in OA research. We review the current role of radiography, as well as advances in quantitative three-dimensional morphological cartilage assessment and semiquantitative whole-organ assessment of OA. Although magnetic resonance imaging has evolved as the leading imaging method in OA research, recent developments in computed tomography are also discussed briefly. Finally, we address the experience from the Foundation for the National Institutes of Health Biomarker Consortium biomarker qualification study and the future role of artificial intelligence.

show abstract

The 16th International Workshop on Osteoarthritis Imaging: “New insights on treatment and imaging: A couple that belongs together”

Sekiya

Ozeki

2022

Osteoarthritis Imaging

View full text Add to dashboard Cite

Comparison of Various Metrics for Evaluating the Performance of Deep Learning Binary Classification, Particularly When Underlying Imaging Data Are Imbalanced

Cited by 3 publications

References 0 publications

ShapeMed-Knee: A Dataset and Neural Shape Model Benchmark for Modeling 3D Femurs

ShapeMed-Knee: A Dataset and Neural Shape Model Benchmark for Modeling 3D Femurs

Imaging Biomarkers of Osteoarthritis

The 16th International Workshop on Osteoarthritis Imaging: “New insights on treatment and imaging: A couple that belongs together”

Contact Info

Product

Resources

About