Learning structured and non-redundant representations with deep neural networks

Yang, Jihai; Xiong, Wei; Li, Shijun; Xu, Chang

doi:10.1016/j.patcog.2018.08.017

Cited by 21 publications

(4 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These diverse representations are then used as an input of the model to improve performance [172], [173]. Now CNN is an effective feature learner that can automatically extract discriminating features depending upon the problem [174].…”

Section: Channel(input) Exploitation Based Cnnsmentioning

confidence: 99%

A survey of the recent architectures of deep convolutional neural networks

et al. 2020

View full text Add to dashboard Cite

Deep Convolutional Neural Networks (CNNs) are a special type of Neural Networks, which have shown exemplary performance on several competitions related to Computer Vision and Image Processing. Interesting application areas of CNN include Image Classification and Segmentation, Object Detection, Video Processing, Natural Language Processing, Speech Recognition, etc. The powerful learning ability of deep CNN is largely due to the use of multiple feature extraction stages that can automatically learn representations from the data. Availability of a large amount of data and improvements in the hardware technology have accelerated the research in CNNs, and recently very interesting deep CNN architectures have been reported. In fact, several interesting ideas to bring advancements in CNNs have been explored such as the use of different activation and loss functions, parameter optimization, regularization, and architectural innovations. However, the major improvement in representational capacity of the deep CNN is achieved through architectural innovations. Especially, the idea of exploiting spatial and channel information, depth and width of architecture, and multi-path information processing has gained substantial attention. Similarly, the idea of using a block of layers as a structural unit is also gaining popularity. This survey thus focuses on the intrinsic taxonomy present in the recently reported deep CNN architectures and consequently, classifies the recent innovations in CNN architectures into seven different categories. These seven categories are based on spatial exploitation, depth, multi-path, width, feature-map exploitation, channel boosting, and attention. Additionally, the elementary understanding of CNN components, current challenges and applications of CNN are also provided. CNNs are the best among learning algorithms in understanding images content, and have shown exemplary results in segmentation, classification, detection, and retrieval related tasks [8], [9]. The success of CNNs has captured attention beyond academia. In industry, companies such as Google, Microsoft, AT&T, NEC, and Facebook have developed active research groups for exploring new architectures of CNN [10]. At present, most of the frontrunners of image processing and computer vision competitions are employing deep CNN based models.The attractive feature of CNN is its ability to exploit spatial or time correlation of the data. The topology of CNN is divided into multiple learning stages composed of a combination of the convolutional layers, non-linear processing units, and subsampling layers [11]. CNNs are feedforward multilayered hierarchical networks that are similar to fully connected neural network where each layer, using a bank of convolutional kernels, performs multiple transformations [12]. Convolution operation extracts useful features from locally correlated data points. Output of the convolutional kernels is assigned to non-linear processing unit (activation function), which not only helps in learning abstractions but also emb...

show abstract

Section: Channel(input) Exploitation Based Cnnsmentioning

confidence: 99%

A survey of the recent architectures of deep convolutional neural networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Another negative aspect of increasing the number of layers in deep models may reflect the occurrence of overfitting, especially in lack of data diversity [32]. Nevertheless, using convolution optimization techniques may solve the overfitting problem by optimally select the proper parameters of the deep model according to the problem at hand [46]. In addition, with the emerging progress in hardware and learning algorithms, there are many remedies for the expensive computation cost of deep learning models [9,25].…”

Section: =mentioning

confidence: 99%

Energy time series forecasting-analytical and empirical assessment of conventional and machine learning models

Hamdoun

Sagheer

Youness

2021

IFS

View full text Add to dashboard Cite

Machine learning methods have been adopted in the literature as contenders to conventional methods to solve the energy time series forecasting (TSF) problems. Recently, deep learning methods have been emerged in the artificial intelligence field attaining astonishing performance in a wide range of applications. Yet, the evidence about their performance in to solve the energy TSF problems, in terms of accuracy and computational requirements, is scanty. Most of the review articles that handle the energy TSF problem are systematic reviews, however, a qualitative and quantitative study for the energy TSF problem is not yet available in the literature. The purpose of this paper is twofold, first it provides a comprehensive analytical assessment for conventional, machine learning, and deep learning methods that can be utilized to solve various energy TSF problems. Second, the paper carries out an empirical assessment for many selected methods through three real-world datasets. These datasets related to electrical energy consumption problem, natural gas problem, and electric power consumption of an individual household problem. The first two problems are univariate TSF and the third problem is a multivariate TSF. Compared to both conventional and machine learning contenders, the deep learning methods attain a significant improvement in terms of accuracy and forecasting horizons examined. In the meantime, their computational requirements are notably greater than other contenders. Eventually, the paper identifies a number of challenges, potential research directions, and recommendations to the research community may serve as a basis for further research in the energy forecasting domain.

show abstract

“…(Rahimzadeh et al, 2016) or SVM (Kheirandish et al, 2016). Use of AI-based methods have now exploded with use of methods like artificial neural networks both feed-forward and recurrent (Jeppesen et al, 2017;Zhao et al, 2019;Ma et al, 2018), extreme and deep learning methods (Yang et al, 2019b;Liu et al, 2019), clustering methods including support vector and k-means (Liu et al, 2018). The black-box methods are better suited for complex non-linear systems, but they require a large amount of experimental data (Petrone et al, 2013) and only the SVM-based ones are characterised by a high level of genericity.…”

Section: Related Workmentioning

confidence: 99%

Enhancing polymer electrolyte membrane fuel cell system diagnostics through semantic modelling

Tsalapati

Johnson

Jackson

et al. 2021

Expert Systems with Applications

View full text Add to dashboard Cite

Polymer electrolyte membrane fuel cells (PEMFC) are a promising technology for economic and environmentally friendly energy production. However, they haven't reached their full potential in the market yet as only few reliable PEMFC systems have successfully passed the prototyping face. A drawback of the current diagnostic tools is that only a select few are of high genericity, reliability and can perform efficiently on-line at the same time. Furthermore, there is only limited research identifying both PEMFC stack faults and ancillary system faults simultaneously. While none of the existing tools can be interrogated by the end-user. In this research, we develop novel artificial intelligence-based technologies to overcome these existing barriers, i.e., i) a semantically enriched integrating schema (ontology) of the overall operation and structure of the PEMFC that allows automatic inference engines to automatically deduce fault detection; ii) a knowledgebased, light-weight, on-line fuel cell system diagnosis (FuCSyDi) platform. FuCSyDi detects and provides the location of failures by considering only the data from the reliable sensors. Additionally, it provides the reasons underpinning any forthcoming failures and enables the end-user to interrogate the platform for further information regarding its operation and structure. Our platform is validated by performing tests against common automotive stress conditions. This innovative approach enhances the reliability of the fuel cell system diagnosis and, hence, its lifetime performance.

show abstract

Learning structured and non-redundant representations with deep neural networks

Cited by 21 publications

References 19 publications

A survey of the recent architectures of deep convolutional neural networks

A survey of the recent architectures of deep convolutional neural networks

Energy time series forecasting-analytical and empirical assessment of conventional and machine learning models

Enhancing polymer electrolyte membrane fuel cell system diagnostics through semantic modelling

Contact Info

Product

Resources

About