Learning-Based Cost Functions for 3-D and 4-D Multi-Surface Multi-Object Segmentation of Knee MRI: Data From the Osteoarthritis Initiative

Abstract

A fully automated knee magnetic resonance imaging (MRI) segmentation method to study osteoarthritis (OA) was developed using a novel hierarchical set of random forests (RF) classifiers to learn the appearance of cartilage regions and their boundaries. A neighborhood approximation forest is used first to provide contextual feature to the second-level RF classifier that also considers local features and produces location-specific costs for the layered optimal graph image segmentation of multiple objects and surfaces (LOGISMOS) framework. Double-echo steady state MRIs used in this paper originated from the OA Initiative study. Trained on 34 MRIs with varying degrees of OA, the performance of the learning-based method tested on 108 MRIs showed significant reduction in segmentation errors ( ) compared with the conventional gradient-based and single-stage RF-learned costs. The 3-D LOGISMOS was extended to longitudinal-3-D (4-D) to simultaneously segment multiple follow-up visits of the same patient. As such, data from all time-points of the temporal sequence contribute information to a single optimal solution that utilizes both spatial 3-D and temporal contexts. 4-D LOGISMOS validation on 108 MRIs from baseline, and 12 month follow-up scans of 54 patients showed significant reduction in segmentation errors ( ) compared with 3-D. Finally, the potential of 4-D LOGISMOS was further explored on the same 54 patients using five annual follow-up scans demonstrating a significant improvement of measuring cartilage thickness ( ) compared with the sequential 3-D approach.

A review on segmentation of knee articular cartilage: from conventional methods towards deep learning

“…The cost function of the BCI surface is determined by the first-order derivatives of nodes on the bone surface, while the cost function of cartilage is the weighted combination of first and second-order derivatives. In 2018, Kashyap et al [154,155] extended LOGISMOS by integrating a post-processing interaction step known as just-enough interaction (JEI). A hierarchy of random forest (RF) classifiers were used to enhance the location-specific cost functions, where the output probabilities of the second RF (learning-based cost function) serves as the cost function for LOGIS-MOS.…”

“…They claimed a significant reduction in signed error of bone surface positioning for femur bone (0.03 mm), when compared to tibia bone [154]. The learning-based cost function on all cartilage compartments had been assessed [155]. The study reported significant reduction in both signed and unsigned cartilage surface positioning errors (p-value << 0.001), except on the medial tibial cartilage (cMT) (p-value = 0.193).…”

Personalized Knee Geometry Modeling Based on Multi-Atlas Segmentation and Mesh Refinement

“…Specifically, the method in [20] is based on a joint optimization scheme using support vector machines for the description of image features and spatial dependencies and discriminative random fields for the interaction term, while the method in [19] utilizes localized Markov random fields that adaptively emphasize appearance and shape priors, according to local region voxel intensities. Simultaneous segmentation of multiple surfaces, where the segmentation problem is treated as a graph optimization problem, is performed in [22] and recently extended to capture spatiotemporal (longitudinal) context [23]. Contentbased features extracted from the edges and the gray level FIGURE 1.…”

Quantitative 3D Analysis of Coronary Wall Morphology in Heart Transplant Patients: OCT-Assessed Cardiac Allograft Vasculopathy Progression

“…The LOGISMOS (Layered Optimal Graph-based Image Segmentation for Multiple Objects and Surfaces) segmentation approach was introduced in 2006 (Li et al, 2006) and has been continuously developed and improved ever since (Yin et al, 2010; Sun et al, 2013; Oguz and Sonka, 2014; Sonka and Abramoff, 2016; Kashyap et al, 2018). Its brief description is provided here for completeness.…”