Advances in Morphological Neural Networks: Training, Pruning and Enforcing Shape Constraints

Abstract

In this paper, we study an emerging class of neural networks, the Morphological Neural networks, from some modern perspectives. Our approach utilizes ideas from tropical geometry and mathematical morphology. First, we state the training of a binary morphological classifier as a Difference-of-Convex optimization problem and extend this method to multiclass tasks. We then focus on general morphological networks trained with gradient descent variants and show, quantitatively via pruning schemes as well as qualitatively, the sparsity of the resulted representations compared to FeedForward networks with ReLU activations as well as the effect the training optimizer has on such compression techniques. Finally, we show how morphological networks can be employed to guarantee monotonicity and present a softened version of a known architecture, based on Maslov Dequantization, which alleviates issues of gradient propagation associated with its "hard" counterparts and moderately improves performance.

On optimizing morphological neural networks for hyperspectral image classification

“…Dimitriades and Maragos investigated the use of purely morphological architectures in computer vision classification and found that these models performed slightly worse than their convolutional counterparts [29]. This has led to an increase in the use of hybrid morphological-convolution network architectures in various domains [28,[30][31][32][33][34].…”

“…This has led to an increase in the use of hybrid morphological-convolution network architectures in various domains [28,[30][31][32][33][34]. The sparsity induced by replacing linear operators (such as traditional 2D convolution) with morphological ones has been previously examined in the literature [22,29,35].…”