Weight asynchronous update: Improving the diversity of filters in a deep convolutional network

Zhang, Dejun; He, Linchao; Luo, Mengting; Xu, Zhanya; He, Fazhi

doi:10.1007/s41095-020-0185-5

Cited by 5 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, it is robust in the occlusion or undetected joints scenario and generalizes well for the in the wild scenario. In the future work, we will optimize our network [45,46] for better time performance and apply the idea of LCMDN into more extended fields, such as visual tracking [47], multi-view and multi-person pose estimation [46,48], or 3D human hand pose estimation [49].…”

Section: Discussionmentioning

confidence: 99%

An Improved Mixture Density Network for 3D Human Pose Estimation with Ordinal Ranking

Zhang

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Estimating accurate 3D human poses from 2D images remains a challenge due to the lack of explicit depth information in 2D data. This paper proposes an improved mixture density network for 3D human pose estimation called the Locally Connected Mixture Density Network (LCMDN). Instead of conducting direct coordinate regression or providing unimodal estimates per joint, our approach predicts multiple possible hypotheses by the Mixture Density Network (MDN). Our network can be divided into two steps: the 2D joint points are estimated from the input images first; then, the information of human joints correlation is extracted by a feature extractor. After the human pose feature is extracted, multiple pose hypotheses are generated via the hypotheses generator. In addition, to make better use of the relationship between human joints, we introduce the Locally Connected Network (LCN) as a generic formulation to replace the traditional Fully Connected Network (FCN), which is applied to a feature extraction module. Finally, to select the most appropriate 3D pose result, a 3D pose selector based on the ordinal ranking of joints is adopted to score the predicted pose. The LCMDN improves the representation capability and robustness of the original MDN method notably. Experiments are conducted on the Human3.6M and MPII dataset. The average Mean Per Joint Position Error (MPJPE) of our proposed LCMDN reaches 50 mm on the Human3.6M dataset, which is on par or better than the state-of-the-art works. The qualitative results on the MPII dataset show that our network has a strong generalization ability.

show abstract

Section: Discussionmentioning

confidence: 99%

An Improved Mixture Density Network for 3D Human Pose Estimation with Ordinal Ranking

Zhang

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Residual operation is introduced to neural network by He et al [31] to ease the model optimization difficulty and it now has been widely applied to image classification, image dehazing, object detection, etc. [32,33,34,35,36] Compared with the classic residual operation, using parallel RNN module as residual component has following characterstics: 1) Compound residual operation: parallel RNN module is a generalization of the classic residual operation, which adds linear projection and bias to inputs. When all internal parameters are initialized to identity matrix, it degenerates to the classic residual operation.…”

Section: Parallel Rnn Modulementioning

confidence: 99%

Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN

Jiang

Qin

Cao

et al. 2021

Preprint

View full text Add to dashboard Cite

The recurrent network architecture is a widely used model in sequence modeling, but its serial dependency hinders the computation parallelization, which makes the operation inefficient. The same problem was encountered in serial adder at the early stage of digital electronics. In this paper, we discuss the similarities between recurrent neural network (RNN) and serial adder. Inspired by carry-lookahead adder, we introduce carry-lookahead module to RNN, which makes it possible for RNN to run in parallel. Then, we design the method of parallel RNN computation, and finally Carry-lookahead RNN (CL-RNN) is proposed. CL-RNN takes advantages in parallelism and flexible receptive field. Through a comprehensive set of tests, we verify that CL-RNN can perform better than existing typical RNNs in sequence modeling tasks which are specially designed for RNNs. Code and models are available at: https://github.com/WinnieJiangHW/Carry-lookahead_RNN.

show abstract

“…Zhang et al [4] proposed a method for updating a specific subfilter cascade chosen dynamically during training to produce more diverse convolutional filters and reduce overlap in representation. A more general examination of this problem found solutions like structuring a network to resemble the knowledge base, which could be achieved either manually [5] or through a training process [6].…”

Section: Introductionmentioning

confidence: 99%

Detection of Unknown Polymorphic Patterns Using Feature-Extracting Part of a Convolutional Autoencoder

Kucharski,

Ślot

2023

Applied Sciences

View full text Add to dashboard Cite

Background: The present paper proposes a novel approach for detecting the presence of unknown polymorphic patterns in random symbol sequences that also comprise already known polymorphic patterns. Methods: We propose to represent rules that define the considered patterns as regular expressions and show how these expressions can be modeled using filter cascades of neural convolutional layers. We adopted a convolutional autoencoder (CAE) as a pattern detection framework. To detect unknown patterns, we first incorporated knowledge of known rules into the CAE’s convolutional feature extractor by fixing weights in some of its filter cascades. Then, we executed the learning procedure, where the weights of the remaining filters were driven by two different objectives. The first was to ensure correct sequence reconstruction, whereas the second was to prevent weights from learning the already known patterns. Results: The proposed methodology was tested on sample sequences derived from the human genome. The analysis of the experimental results provided statistically significant information on the presence or absence of polymorphic patterns that were not known in advance. Conclusions: The proposed method was able to detect the existence of unknown polymorphic patterns.

show abstract

Weight asynchronous update: Improving the diversity of filters in a deep convolutional network

Cited by 5 publications

References 23 publications

An Improved Mixture Density Network for 3D Human Pose Estimation with Ordinal Ranking

An Improved Mixture Density Network for 3D Human Pose Estimation with Ordinal Ranking

Recurrent Neural Network from Adder's Perspective: Carry-lookahead RNN

Detection of Unknown Polymorphic Patterns Using Feature-Extracting Part of a Convolutional Autoencoder

Contact Info

Product

Resources

About