A Maximum Entropy Deep Reinforcement Learning Neural Tracker

Abstract

Tracking of anatomical structures has multiple applications in the field of biomedical imaging, including screening, diagnosing and monitoring the evolution of pathologies. Semi-automated tracking of elongated structures has been previously formulated as a task for deep reinforcement learning (DRL), albeit it remains a challenge. We introduce a maximum entropy continuous-action DRL neural tracker capable of training from scratch in a complex environment in the presence of high noise levels, Gaussian blurring and cell detractors. The trained model is evaluated on mouse cortical two-photon microscopy images. At the expense of slightly worse robustness compared to a previously applied DRL tracker, we reach significantly higher accuracy, approaching the performance of the standard hand-engineered algorithm used for neuron tracing. The higher sample efficiency of our maximum entropy DRL tracker indicates its potential of being applied directly to small biomedical datasets in the absence of artificial models.

A deep Koopman operator‐based modelling approach for long‐term prediction of dynamics with pixel‐level measurements

“…In our work, the encoder of CKNet is implemented separately using deterministic and variational convolutional networks (DCKNet and VCKNet), in which spectral analysis is performed to study the relationship between them on spanning the latent state space to approximate the Koopman operator. Four Mujoco cases are modelled by DCKNet and are trained with soft actor-critic (SAC) [44] simultaneously. Two offline cases in OpenAI Gym (Gym for brevity in the rest of this paper) are modelled based on pre-collected datasets.…”

“…For offline training, all cases are trained with DCKNet and VCKNet separately. To implement back-propagation in VCKNet, the reparameterised technique [44,55] is applied to sample observables in the training process, that is…”

“…For online training cases in Mujoco, soft actor-critic (SAC) [44] is adopted to provide training data for DCKNet constantly along with the entire training process. The encoder is updated synchronously with the critic and the Koopman operator, while the decoder is only updated with the Koopman operator.…”

“…β ⋆ means corresponding learning rates of weights ⋆, where ⋆ ∈ fzc; K; zag. Algorithm 2 points out the updating order of all neural networks, and more details of SAC refer to [44].…”

Neuron tracing from light microscopy images: automation, deep learning and bench testing

“…This approach was further improved by PNR ( Radojević and Meijering, 2017b , 2019 ) and PAT ( Skibbe et al , 2019 ) using Monte Carlo filtering. Zhang et al (2018) , Dai et al (2019) and Balaram et al (2019) reformulated the tracing as a behavior problem and introduced a deep reinforcement learning strategy to guide the tracing process. Athey et al (2022) connected the broken components traced by the Bayesian appearance imaging model employing a hidden Markov model.…”