Improving generalization of deep neural networks by leveraging margin distribution

A Region‐Based Analysis for the Feature Concatenation in Deep Forests

“…Lyu e t al . [ 32 ] reformulate deep forest as an additive model boosting new features by optimizing the margin distribution layer by layer. They first give a theoretical explanation of the success of cascade structure from the perspective of margin theory.…”

“…Lots of work has recently improved the two components of deep forests [ 29,31,32,37 ] and expanded the tree‐based deep models to some specific settings, such as multi‐label learning [ 26 ] , multi‐instance learning [ 38 ] , multi‐modal learning [ 39 ] , semi‐supervised learning [ 27 ] , and crowdsourcing aggregation [ 40 ] . It would be interesting to explore the possibility of exploiting deep forests for rehearsal [ 41 ] .…”

“…Recently, Lyu et al . [ 32 ] give a theoretical explanation of the success of a boosting‐type feature refinement from the perspective of margin theory. Pang et al .…”

“…Lyu et al . [ 32 ] and Arnould et al . [ 33 ] try to analyze the generalization performance from the perspective of variance reducing in some simplified cases.…”

“…Deep neural networks (DNNs) have recently demonstrated their superiority in machine learning and have been successfully applied to a variety of tasks, including computer vision (CV) [15,16] , automatic speech recognition (ASR) [17] , and natural language processing (NLP) [18] . They are composed of parameterized differentiable non-linear modules trained by the backpropagation procedure.…”