Harvesting, Detecting, and Characterizing Liver Lesions from Large-scale Multi-phase CT Data via Deep Dynamic Texture Learning

Abstract

Effective and non-invasive radiological imaging based tumor/lesion characterization (e.g., subtype classification) has long been a major aim in the oncology diagnosis and treatment procedures, with the hope of reducing needs for invasive surgical biopsies. Prior work are generally very restricted to a limited patient sample size, especially using patient studies with confirmed pathological reports as ground truth. In this work, we curate a patient cohort of 1305 dynamic contrast CT studies (i.e., 5220 multi-phase 3D volumes) with pathology confirmed ground truth. A novel fully-automated and multi-stage liver tumor characterization framework is proposed, comprising four steps of tumor proposal detection, tumor harvesting, primary tumor site selection, and deep texture-based characterization. More specifically, (1) we propose a 3D non-isotropic anchor-free lesion detection method; (2) we present and validate the use of multi-phase deep texture learning for precise liver lesion tissue characterization, named spatially adaptive deep texture (SaDT);(3) we leverage small-sized public datasets to semi-automatically curate our large-scale clinical dataset of 1305 patients where four main liver tumor subtypes of primary, secondary, metastasized and benign are presented. Extensive evaluations demonstrate that our new data curation strategy, combined with the SaDT deep dynamic texture analysis, can effectively improve the mean F1 scores by > 8.6% compared with baselines, in differentiating four major liver lesion types. This is a significant step towards the clinical goal.

Fully-Automated Liver Tumor Localization and Characterization from Multi-Phase MR Volumes Using Key-Slice ROI Parsing: A Physician-Inspired Approach

“…As demonstrated in Table 1, compared with purely using classification networks, KSP significantly improves accuracy (+5%), mean F1 (+9%-12%) and HCC F1 scores (+2%-10%). SaDT [16] garners the highest HCC vs. others F1 score of 0.804, which is comparable to reported physician performance (0.791) [2]. Traditional radiomics method usually under-performs deep learning methods with a large margin, especially compared to our method.…”

“…For the LBOS, the standard UNet [19] network with distance-aware Tversky loss is implemented. As for lesion characterization, we test three standard classifiers, ResNet101 [13], DenseNet121 [15] and ResNeXt101 [24], as well as two texture based classifiers, DeepTEN [28] and SaDT [16].…”

“…Prior CAD studies, except for Zhen et al [30], also only focus on computed tomography (CT), despite the greater promise of MR. Most importantly, apart from two studies [7,16], CAD works typically assume a manually drawn region of interest (ROI) is available. In doing so, they elide the major challenge of parsing a medical volume to determine diagnosable ROIs.…”

“…Without this capability, manual intervention remains necessary and the system also remains susceptible to interuser variations. The most obvious localization strategy follows computer vision practices and directly applies a detector, e.g., [16]. However, detectors aim to find all lesions and their entire 3D extent in a study, whereas the needs for liver lesion characterization are distinct: reliably localize the key diagnosable ROI(s).…”

“…1(a) depicts these three types. To localize key diagnosable ROIs, we use a physician-inspired approach that departs from standard detection frameworks seen in computer vision and used elsewhere [16]. Instead, we propose a keyslice parser (KSP), which breaks down the parsing problem similarly to clinical practice, e.g., the ubiquitous response evaluation criteria in solid tumors (RECIST) protocol [9].…”