EffNet: An Efficient Structure for Convolutional Neural Networks

Freeman, Ido; Roese-Koerner, Lutz; Kummert, Anton

doi:10.48550/arxiv.1801.06434

Cited by 5 publications

(4 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For that, there are two possible categories to work on: (1) Classification NN that should be modified to perform regression and re-trained [28] and (2) Regression NNs [29]. Although the classification accuracy achieved by the one-sided Jacobi TSVM could be obtained by an existing NN model from the two above mentioned categories, the main concern remains in the computational Even the smallest models such as MobileNet [30], Shuffle Net [31], and EffNet [32] contains at least four layers. On the other hand, for the regression NNs, with only one hidden layer, a shallow network is considered to be the smallest possible regression NN model.…”

Section: A Network Structurementioning

confidence: 99%

A Shallow Neural Network for Real-Time Embedded Machine Learning for Tensorial Tactile Data Processing

Younes

Ibrahim

Rizk

et al. 2021

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

This paper presents a novel hardware architecture of the Tensorial Support Vector Machine (TSVM) based on Shallow Neural Networks (NN) for the Single Value Decomposition (SVD) computation. The proposed NN achieves a comparable Mean Squared Error and Cosine Similarity to the widely used one-sided Jacobi algorithm. When implemented on an FPGA, the NN offers 324× faster computations than the one-sided Jacobi with reductions up to 58% and 67% in terms of hardware resources and power consumption respectively. When validated on a touch modality classification problem, the NN-based TSVM implementation has achieved a real-time operation while consuming about 88% less energy per classification than the Jacobi-based TSVM with an accuracy loss of at most 3%. Such results offer the ability to deploy intelligence on resource-limited platform for energy-constrained applications.

show abstract

Section: A Network Structurementioning

confidence: 99%

A Shallow Neural Network for Real-Time Embedded Machine Learning for Tensorial Tactile Data Processing

Younes

Ibrahim

Rizk

et al. 2021

IEEE Trans. Circuits Syst. I

View full text Add to dashboard Cite

show abstract

“…Since most of the computation in deep networks is due to convolutions, it is logical to focus on reducing the computational burden due to them. Howard et al (2017); Zhang et al (2017); Freeman et al (2018) employ variations of convolutional layers by taking advantage of the separability of the kernels either by convolving with a group of input channels or by splitting the kernels across the dimensions. In addition to depth-wise convolutions, MobileNetV2 (Sandler et al, 2018) uses inverted residual structures to learn how to combine the inputs in a residual block.…”

Section: Related Workmentioning

confidence: 99%

Self-Binarizing Networks

Lahoud,

Achanta,

Márquez-Neila

et al. 2019

Preprint

View full text Add to dashboard Cite

We present a method to train self-binarizing neural networks, that is, networks that evolve their weights and activations during training to become binary. To obtain similar binary networks, existing methods rely on the sign activation function. This function, however, has no gradients for non-zero values, which makes standard backpropagation impossible. To circumvent the difficulty of training a network relying on the sign activation function, these methods alternate between floating-point and binary representations of the network during training, which is sub-optimal and inefficient. We approach the binarization task by training on a unique representation involving a smooth activation function, which is iteratively sharpened during training until it becomes a binary representation equivalent to the sign activation function. Additionally, we introduce a new technique to perform binary batch normalization that simplifies the conventional batch normalization by transforming it into a simple comparison operation. This is unlike existing methods, which are forced to the retain the conventional floating-point-based batch normalization. Our binary networks, apart from displaying advantages of lower memory and computation as compared to conventional floating-point and binary networks, also show higher classification accuracy than existing state-of-the-art methods on multiple benchmark datasets.

show abstract

“…Yet another variation of MobileNet building block is the EffNet architecture [34], motivated by a careful study. EffNet utilizes depthwise separable convolutions, and pushes it even further by splitting 3£3 convolution into pair of convolutions with 1£3 and 3£1 kernels.…”

Section: Speeding-up Convolutional Neural Networkmentioning

confidence: 99%

Bulletin of the Polish Academy of Sciences: Technical Sciences

Lebedev¹,

Lempitsky²

2018

View full text Add to dashboard Cite

Convolutional neural networks (CNN) have become ubiquitous in computer vision as well as several other domains, but the sheer size of the modern CNNs means that for the majority of practical applications, a significant speed up and compression are often required. Speeding-up CNNs therefore have become a very active area of research with multiple diverse research directions pursued by many groups in academia and industry. In this short survey, we cover several research directions for speeding up CNNs that have become popular recently. Specifically, we cover approaches based on tensor decompositions, weight quantization, weight pruning, and teacher-student approaches. We also review CNN architectures designed for optimal speed and briefly consider automatic architecture search.

show abstract

EffNet: An Efficient Structure for Convolutional Neural Networks

Cited by 5 publications

References 6 publications

A Shallow Neural Network for Real-Time Embedded Machine Learning for Tensorial Tactile Data Processing

A Shallow Neural Network for Real-Time Embedded Machine Learning for Tensorial Tactile Data Processing

Self-Binarizing Networks

Bulletin of the Polish Academy of Sciences: Technical Sciences

Contact Info

Product

Resources

About