A New Approach to Neural Network Design for Fast Convergence via Feed-forward Loop

Abstract

A feed-forward loop has been widely used in control systems to boost the performance without hurting the overall stability of the system. We propose a new neural network, FfcNet by adopting the idea from the control theory. The proposed network adds a pre-designed feed-forward loop in parallel with the existing regular blocks, which is similar to the identity mapping in ResNet network. The feed-forward loop helps the overall network to converge faster while keeping overall system stability and accuracy. It is also shown that the feed-forward loop is equivalent to setting a proper initial condition of the parameters in the network. A special dataset of highly distorted 7-segment LED images is prepared to evaluate the performance of the pattern recognition algorithm. We demonstrated the performance of the proposed design through simulations and found the new design improved the convergence rate by 52% from the original ResNet network while keeping the same test accuracy.