Speech recognition with deep recurrent neural networks

Graves, Alex; Mohamed, Abdelrahman; Hinton, Geoffrey E.

doi:10.1109/icassp.2013.6638947

Cited by 7,175 publications

(4,124 citation statements)

References 20 publications

Supporting

Mentioning

4,082

Contrasting

Unclassified

Order By: Relevance

“…However, growing evidence indicates that sequence specificities can be more accurately captured by more complex techniques [2][3][4][5] . Recently, 'deep learning' has achieved record-breaking performance in a variety of information technology applications 6,7 . We adapted deep learning methods to the task of predicting sequence specificities and found that they compete favorably with the state of the art.…”

Section: A N a Ly S I Smentioning

confidence: 99%

Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning

et al. 2015

View full text Add to dashboard Cite

Knowing the sequence specificities of DNA-and RNA-binding proteins is essential for developing models of the regulatory processes in biological systems and for identifying causal disease variants. Here we show that sequence specificities can be ascertained from experimental data with 'deep learning' techniques, which offer a scalable, flexible and unified computational approach for pattern discovery. Using a diverse array of experimental data and evaluation metrics, we find that deep learning outperforms other state-of-the-art methods, even when training on in vitro data and testing on in vivo data. We call this approach DeepBind and have built a stand-alone software tool that is fully automatic and handles millions of sequences per experiment. Specificities determined by DeepBind are readily visualized as a weighted ensemble of position weight matrices or as a 'mutation map' that indicates how variations affect binding within a specific sequence.DNA-and RNA-binding proteins play a central role in gene regulation, including transcription and alternative splicing. The sequence specificities of a protein are most commonly characterized using position weight matrices 1 (PWMs), which are easy to interpret and can be scanned over a genomic sequence to detect potential binding sites. However, growing evidence indicates that sequence specificities can be more accurately captured by more complex techniques 2-5 . Recently, 'deep learning' has achieved record-breaking performance in a variety of information technology applications 6,7 . We adapted deep learning methods to the task of predicting sequence specificities and found that they compete favorably with the state of the art. Our approach, called DeepBind, is based on deep convolutional neural networks and can discover new patterns even when the locations of patterns within sequences are unknown-a task for which traditional neural networks require an exorbitant amount of training data.There are several challenging aspects in learning models of sequence specificity using modern high-throughput technologies. First, the data come in qualitatively different forms. Protein binding microarrays (PBMs) 8 and RNAcompete assays 9 provide a specificity coefficient for each probe sequence, whereas chromatin immunoprecipitation (ChIP)-seq 10 provides a ranked list of putatively bound sequences of varying length, and HT-SELEX 11 generates a set of very high affinity sequences. Second, the quantity of data is large. A typical high-throughput experiment measures between 10,000 and 100,000 sequences, and it is computationally demanding to incorporate them all. Third, each data acquisition technology has its own artifacts, biases and limitations, and we must discover the pertinent specificities despite these unwanted effects. For example, ChIP-seq reads often localize to "hyper-ChIPable" regions of the genome near highly expressed genes 12 .DeepBind (Fig. 1) addresses the above challenges. (i) It can be applied to both microarray and sequencing data; (ii) it can learn from millions of...

show abstract

Section: A N a Ly S I Smentioning

confidence: 99%

Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Neural networks have become increasingly popular in recent years because of their tremendous success in image classification [1,2], speech recognition [3,4] and natural language processing tasks [5,6]. In fact, deep learning methods have regularly won many recent challenges in these domains [3,7].…”

Section: Introductionmentioning

confidence: 99%

Using a Random Forest to Inspire a Neural Network and Improving on It

Wang¹,

Aggarwal²,

Huan³

2017

Proceedings of the 2017 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Neural networks have become very popular in recent years because of the astonishing success of deep learning in various domains such as image and speech recognition. In many of these domains, specific architectures of neural networks, such as convolutional networks, seem to fit the particular structure of the problem domain very well, and can therefore perform in an astonishingly effective way. However, the success of neural networks is not universal across all domains. Indeed, for learning problems without any special structure, or in cases where the data is somewhat limited, neural networks are known not to perform well with respect to traditional machine learning methods such as random forests. In this paper, we show that a carefully designed neural network with random forest structure can have better generalization ability. In fact, this architecture is more powerful than random forests, because the back-propagation algorithm reduces to a more powerful and generalized way of constructing a decision tree. Furthermore, the approach is efficient to train and requires a small constant factor of the number of training examples. This efficiency allows the training of multiple neural networks in order to improve the generalization accuracy. Experimental results on 10 realworld benchmark datasets demonstrate the effectiveness of the proposed enhancements.

show abstract

“…Long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997) models solve this problem by introducing different gate functions that control the flow of information. This approach has been quite successful in tasks involving language models (Sutskever et al, 2014) and speech recognition (Graves et al, 2013). Meanwhile, breakthroughs in image captioning (Vinyals et al, 2015) have succeeded in transforming images into the language domain by combining the convolutional neural network (CNN) and recurrent models.…”

Section: Trends In the Development Of Ai Technology Applications Formentioning

confidence: 99%

Current trends in the development of intelligent unmanned autonomous systems

Zhang

et al. 2017

Frontiers Inf Technol Electronic Eng

134

View full text Add to dashboard Cite

Intelligent unmanned autonomous systems are some of the most important applications of artificial intelligence (AI). The development of such systems can significantly promote innovation in AI technologies. This paper introduces the trends in the development of intelligent unmanned autonomous systems by summarizing the main achievements in each technological platform. Furthermore, we classify the relevant technologies into seven areas, including AI technologies, unmanned vehicles, unmanned aerial vehicles, service robots, space robots, marine robots, and unmanned workshops/intelligent plants. Current trends and developments in each area are introduced.

show abstract

Speech recognition with deep recurrent neural networks

Cited by 7,175 publications

References 20 publications

Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning

Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning

Using a Random Forest to Inspire a Neural Network and Improving on It

Current trends in the development of intelligent unmanned autonomous systems

Contact Info

Product

Resources

About