Globally optimal OCT surface segmentation using a constrained IPM optimization

Abstract

Segmentation of multiple surfaces in optical coherence tomography (OCT) images is a challenging problem, further complicated by the frequent presence of weak boundaries, varying layer thicknesses, and mutual influence between adjacent surfaces. The traditional graph-based optimal surface segmentation method has proven its effectiveness with its ability to capture various surface priors in a uniform graph model. However, its efficacy heavily relies on handcrafted features that are used to define the surface cost for the “goodness” of a surface. Recently, deep learning (DL) is emerging as a powerful tool for medical image segmentation thanks to its superior feature learning capability. Unfortunately, due to the scarcity of training data in medical imaging, it is nontrivial for DL networks to implicitly learn the global structure of the target surfaces, including surface interactions. This study proposes to parameterize the surface cost functions in the graph model and leverage DL to learn those parameters. The multiple optimal surfaces are then simultaneously detected by minimizing the total surface cost while explicitly enforcing the mutual surface interaction constraints. The optimization problem is solved by the primal-dual interior-point method (IPM), which can be implemented by a layer of neural networks, enabling efficient end-to-end training of the whole network. Experiments on spectral-domain optical coherence tomography (SD-OCT) retinal layer segmentation demonstrated promising segmentation results with sub-pixel accuracy.

A deep learning network with differentiable dynamic programming for retina OCT surface segmentation

“…As in the IPM segmentation method [31], we use image gradient information as additional input channels to enrich image input information while reducing the learning burden of the network, as image gradients are prominent features to discriminate image boundaries. Our proposed network used seven gradient channels including gradient scales along the orientations of 0 • (x-dimension), 45 • , 90 • , and 135 • , normalized gradient directions in the 0 • -90 • and 45 • -135 • coordinate systems, and the gradient magnitude in the 0 • -90 • coordinate system, all of which are directly computed from raw images.…”

“…Armed with superior data representation learning capacity, deep learning (DL) methods are emerging as powerful alternatives to traditional segmentation algorithms for many medical image segmentation tasks [15,28]. Fully convolutional networks (FCNs) [24,18], Convolutional neural networks (CNNs) [27], and U-Net [23,13,8,16,31] have been utilized for retinal layer segmentation in OCT images. Due to the scarcity of training data in medical imaging, it is yet nontrivial for DL networks to implicitly learn global structures of the target surfaces.…”

“…The hybrid 2D-3D CNN [16] using B-scan alignment was proposed to obtain continuous 3D retinal layer surfaces from OCT. The IPM optimization method [31] effectively integrates the DL feature learning with the IPM optimization to enforce mutual interaction between surfaces, but the IPM optimization runs on each A-scan. All these methods [8,16,31] achieved highly accurate segmentation of retinal surfaces from OCT.…”

“…The IPM optimization method [31] effectively integrates the DL feature learning with the IPM optimization to enforce mutual interaction between surfaces, but the IPM optimization runs on each A-scan. All these methods [8,16,31] achieved highly accurate segmentation of retinal surfaces from OCT. However, their performance is prone to be affected by image outliers with bad quality or artifacts with the limited size of training data [8], as they lack the capability of explicitly learning surface smoothness structure.…”

“…This study proposes to unify the powerful feature learning capability of DL with a constrained DDP module in a single deep neural network for end-toend training to achieve globally optimal segmentation while explicitly enforcing surface smoothness. In the proposed segmentation framework, a U-Net with additional image gradient channels [31] is leveraged as the backbone for learning parameterized surface costs. The retinal surface inference by minimizing the total surface cost while satisfying surface smoothness constraints is realized by a DDP module for a globally optimal solution.…”