Input layer regularization for magnetic resonance relaxometry biexponential parameter estimation

Abstract

Many methods have been developed for estimating the parameters of biexponential decay signals, which arise throughout magnetic resonance relaxometry (MRR) and the physical sciences. This is an intrinsically ill‐posed problem so that estimates can depend strongly on noise and underlying parameter values. Regularization has proven to be a remarkably efficient procedure for providing more reliable solutions to ill‐posed problems, while, more recently, neural networks have been used for parameter estimation. We re‐address the problem of parameter estimation in biexponential models by introducing a novel form of neural network regularization which we call input layer regularization (ILR). Here, inputs to the neural network are composed of a biexponential decay signal augmented by signals constructed from parameters obtained from a regularized nonlinear least‐squares estimate of the two decay time constants. We find that ILR results in a reduction in the error of time constant estimates on the order of 15%–50% or more, depending on the metric used and signal‐to‐noise level, with greater improvement seen for the time constant of the more rapidly decaying component. ILR is compatible with existing regularization techniques and should be applicable to a wide range of parameter estimation problems.

Emergence of the SVD as an interpretable factorization in deep learning for inverse problems

“…We provide a more general version of the theorem in the Supplementary Information; here the choice of D as a finite difference stencil was taken to align the result with the experimental setup used in [1] Figure 1. We sampled X in Equation 5 from a standard Gaussian ensemble and computed the left singular vectors of the first layer pre-and post-descrambling of the network in [30]. Note that the left singular vectors of the descrambled weight matrix are perfectly oscillating: this is because X as a standard Gaussian ensemble together with a linear wiretapped layer yields a descrambler P (N ) ≈ TrU , which results in P W ≈ TrΣV so that the left singular vectors of the descrambled weight matrix are given by Tr…”

“…The NN with the concatenated native and smoothed versions of the decay curve is termed (ND, Reg), with Reg indicating the smooth decay generated by the regularized nonlinear least squares analysis. This strategy of training on both noisy and smooth data is termed input layer regularization, and improves parameter estimation by 5-10 percent as compared to the more conventional NN estimation of parameters from noisy decay curves [30]. We find that the right singular vectors corresponding to the largest singular values of the first layer are biexponential curves, so that the network learns an input signal library in the class of its training data.…”

“…We demonstrate this phenomenon for a NN in which the forward map s is non-linear, so that the nature of matrices U and V in (1) is entirely unspecified. In particular, following [30] we train a four layer fully-connected NN to learn the exponential parameters (T 2,1 , T 2,2 ) generating a biexponential model:…”

“…The recovery of exponential parameters T 2,1 , T 2,2 from a noisy decay curve is a central problem in magnetic resonance relaxometry [40], and is well-known to be ill-posed with parameter estimates strongly dependent on the noise. This problem has been investigated with a number of neural network-based approaches [41,42]; the novelty of the network in [30] is that it is trained to solve this problem on both noisy and smooth forms of the same data, as a form of input data transformation to incorporate high-fidelity and high-stability and generalizability characteristics into the solutions. To achieve this, the noisy input data is first processed with regularized non-linear least squares parameter estimation, with these estimates used to generate smooth decay curves.…”

“…The concatenated data, now of 128 measurements, was fed through the network with RMSE loss to recover the original (T 2,1 , T 2,2 ). Following the recommendations in [30] we used a Tikhonov regularization procedure in the NLLS pre-processing step with λ = 1.6 × 10 −4 . We also trained the same architecture by concatentating the same sample of noisy data into a size 128 vector; this architecture is termed (ND,ND) and it represents the traditional neural-network approach of recovering parameters from noisy samples of a signal.…”