Factorized Convolutional Neural Networks

Wang, Min; Liu, Baoyuan; Foroosh, Hassan

doi:10.1109/iccvw.2017.71

Cited by 168 publications

(108 citation statements)

References 7 publications

Supporting

Mentioning

108

Contrasting

Order By: Relevance

“…To speed up the computation and reduce the model size, Gonda et al [19] proposed a novel strategy via replacing 3D convolution layers with pairs of 2D and 1D convolution layers. Numerous other attempts on depthwise separable convolutions have been made in various fashions to improve the efficiency of convolution [20][21][22]. Zhang et al [23] combined depthwise separable convolution and spatial separable convolution for liver tumour segmentation.…”

Section: Related Workmentioning

confidence: 99%

3D Dense Separated Convolution Module for Volumetric Medical Image Analysis

Zou

2020

Applied Sciences

View full text Add to dashboard Cite

With the thriving of deep learning, 3D Convolutional Neural Networks have become a popular choice in volumetric image analysis due to their impressive 3D contexts mining ability. However, the 3D convolutional kernels will introduce a significant increase in the amount of trainable parameters. Considering the training data is often limited in biomedical tasks, a tradeoff has to be made between model size and its representational power. To address this concern, in this paper, we propose a novel 3D Dense Separated Convolution (3D-DSC) module to replace the original 3D convolutional kernels. The 3D-DSC module is constructed by a series of densely connected 1D filters. The decomposition of 3D kernel into 1D filters reduces the risk of over-fitting by removing the redundancy of 3D kernels in a topologically constrained manner, while providing the infrastructure for deepening the network. By further introducing nonlinear layers and dense connections between 1D filters, the network's representational power can be significantly improved while maintaining a compact architecture. We demonstrate the superiority of 3D-DSC on volumetric image classification and segmentation, which are two challenging tasks often encountered in biomedical image computing.

show abstract

Section: Related Workmentioning

confidence: 99%

3D Dense Separated Convolution Module for Volumetric Medical Image Analysis

Zou

2020

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Flattened networks [17] construct a network out of fully factorized convolutions and presented the potential of exceptionally factorized networks. Factorized Networks [18] presents a similar factorized convolution in addition to use of topological networks. Successively, the Xception network [19] [21] and deep fried convnets [22].…”

Section: Mobilenet Architecturementioning

confidence: 99%

Efficient mobilenet architecture as image recognition on mobile and embedded devices

Khasoggi

Ermatita

Samsuryadi

2019

IJEECS

View full text Add to dashboard Cite

The introduction of a modern image recognition that has millions of parameters and requires a lot of training data as well as high computing power that is hungry for energy consumption so it becomes inefficient in everyday use. Machine Learning has changed the computing paradigm, from complex calculations that require high computational power to environmentally friendly technologies that can efficiently meet daily needs. To get the best training model, many studies use large numbers of datasets. However, the complexity of large datasets requires large devices and requires high computing power. Therefore large computational resources do not have high flexibility towards the tendency of human interaction which prioritizes the efficiency and effectiveness of computer vision. This study uses the Convolutional Neural Networks (CNN) method with MobileNet architecture for image recognition on mobile devices and embedded devices with limited resources with ARM-based CPUs and works with a moderate amount of training data (thousands of labeled images). As a result, the MobileNet v1 architecture on the ms8pro device can classify the caltech101 dataset with an accuracy rate 92.4% and 2.1 Watt power draw. With the level of accuracy and efficiency of the resources used, it is expected that MobileNet's architecture can change the machine learning paradigm so that it has a high degree of flexibility towards the tendency of human interaction that prioritizes the efficiency and effectiveness of computer vision.

show abstract

“…Numerous methods have been proposed to reduce the computational demands of a deep model by trading prediction accuracy for runtime, via compressing a pre-trained network [6,15,21,24,36,42,47], training small networks directly [11,37], or a combination of both [19]. Using these approaches, a user now needs to decide when to use a specific model, in order to meet the prediction accuracy requirement with minimal latency.…”

Section: Related Workmentioning

confidence: 99%

Adaptive deep learning model selection on embedded systems

Taylor

Marco

Wolff

et al. 2018

Proceedings of the 19th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems

View full text Add to dashboard Cite

The recent groundbreaking advances in deep learning networks (DNNs) make them attractive for embedded systems. However, it can take a long time for DNNs to make an inference on resource-limited embedded devices. Offloading the computation into the cloud is often infeasible due to privacy concerns, high latency, or the lack of connectivity. As such, there is a critical need to find a way to effectively execute the DNN models locally on the devices. This paper presents an adaptive scheme to determine which DNN model to use for a given input, by considering the desired accuracy and inference time. Our approach employs machine learning to develop a predictive model to quickly select a pre-trained DNN to use for a given input and the optimization constraint. We achieve this by first training off-line a predictive model, and then use the learnt model to select a DNN model to use for new, unseen inputs. We apply our approach to the image classification task and evaluate it on a Jetson TX2 embedded deep learning platform using the ImageNet ILSVRC 2012 validation dataset. We consider a range of influential DNN models. Experimental results show that our approach achieves a 7.52% improvement in inference accuracy, and a 1.8x reduction in inference time over the most-capable, single DNN model.

show abstract

Factorized Convolutional Neural Networks

Cited by 168 publications

References 7 publications

3D Dense Separated Convolution Module for Volumetric Medical Image Analysis

3D Dense Separated Convolution Module for Volumetric Medical Image Analysis

Efficient mobilenet architecture as image recognition on mobile and embedded devices

Adaptive deep learning model selection on embedded systems

Contact Info

Product

Resources

About