A generative flow-based model for volumetric data augmentation in 3D deep learning for computed tomographic colonography

Abstract

Purpose Deep learning can be used for improving the performance of computer-aided detection (CADe) in various medical imaging tasks. However, in computed tomographic (CT) colonography, the performance is limited by the relatively small size and the variety of the available training datasets. Our purpose in this study was to develop and evaluate a flow-based generative model for performing 3D data augmentation of colorectal polyps for effective training of deep learning in CADe for CT colonography. Methods We developed a 3D-convolutional neural network (3D CNN) based on a flow-based generative model (3D Glow) for generating synthetic volumes of interest (VOIs) that has characteristics similar to those of the VOIs of its training dataset. The 3D Glow was trained to generate synthetic VOIs of polyps by use of our clinical CT colonography case collection. The evaluation was performed by use of a human observer study with three observers and by use of a CADe-based polyp classification study with a 3D DenseNet. Results The area-under-the-curve values of the receiver operating characteristic analysis of the three observers were not statistically significantly different in distinguishing between real polyps and synthetic polyps. When trained with data augmentation by 3D Glow, the 3D DenseNet yielded a statistically significantly higher polyp classification performance than when it was trained with alternative augmentation methods. Conclusion The 3D Glow-generated synthetic polyps are visually indistinguishable from real colorectal polyps. Their application to data augmentation can substantially improve the performance of 3D CNNs in CADe for CT colonography. Thus, 3D Glow is a promising method for improving the performance of deep learning in CADe for CT colonography.

A review of medical image data augmentation techniques for deep learning applications

“…change of size, shape and location of tumours), 86,119,144,145,152–154,157 which conventional augmentation methods generally do not account for. Sometimes, geometric, deformable and intensity‐based augmentation can also be applied to the data used to train the DL‐based augmentation networks 56,91,101,138,146,152,154 or used alongside the DL‐based methods 53,83,86,87,100,119 . The majority of DL‐based augmentation approaches are based on adversarial training (including GAN‐based and other adversarial learning networks), where a discriminator network is usually used to review the generated images, and the training process iteratively bridges the gap between generated and real images.…”

“…It is difficult to say which type of methods is the best strategy for data augmentation, since the design of the augmentation stage is highly dependent on the given data and tasks. In fact, a hybrid of these types of methods is often used in practice 53,83,86,87,100,119 . For future work, we believe basic and deformable augmentation will continue to be regularly used for training DL networks.…”

“…If an image has one or more contours associated with it, the same transformation is applied to the contours. Geometric transformations are so common that they were utilised by 92 of the 93 basic augmentation studies 15–106 …”

“…Uniform noise, which modifies values by randomly sampling a uniform distribution (3n) or salt and pepper noise, where pixels are randomly set to black or white (3o) have also be used. 25 studies injected noise for data augmentation 24,25,27,30,31,36,39,51,55,57,67,73,75,81,83,87,88,91,93,99,100,103–106 …”

Deep Learning With 3D Convolutional Neural Network for Noninvasive Prediction of Microvascular Invasion in Hepatocellular Carcinoma

“…34 In addition, the 3D CNN is sensitive to directions of the training set. 35 Thus, data augmentation by a simple rotation method was performed on the training set to fill the gap of data's lack to ensure an efficient 3D CNN training. To date, only Jiang et al constructed a predictive model using 3D CNN based on CT images to predict MVI before surgery and the methods achieved an AUC of 0.91.…”