Symbolic Analysis of Classical Neural Networks for Deep Learning

Abstract

Deep learning is based on matrix computing with a large amount of hidden parameters that is not visible outside the computing module. Deep learning is a non-linear system and a linear approach is not possible. It is natural for people to visualize the algorithm and to follow some hidden parameters. In this paper, we propose a simple graphical programming of a nonlinear system based on drawing the simplest unit, such as a single neuron model. A more complex scheme of a classical neural network is obtained by commands copy-move-place of one neuron. The number of neurons and layers can be chosen arbitrarily. Once the scheme is complete, the implementation code is evaluated with symbolic parameters and nonlinear activation functions. This cannot be done manually. With the symbolic expression of outputs in terms of inputs and symbolic parameters, including symbolic activation pure functions, some other properties can be derived in closed form. This unique original approach can help scientists and developers and design numerical algorithms for machine learning and to understand how deep learning algorithms work.