Multilayer perceptron with sparse hidden outputs for phoneme recognition

Sivaram, G.S.V.S.; Heřmanský, Hynek

doi:10.1109/icassp.2011.5947563

Cited by 19 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, there was also the introduction of a new concept based on MLP, the sparse MLP (SMLP) [83,86]. The SMLP are based on the same layout as MLP, with the only difference that the outputs of one of the hidden layers are sparse.…”

Section: Multilayer Perceptronsmentioning

confidence: 99%

“…Hence, the Fig. 4 Multilayer perceptron [83] SOMs are capable of distinguishing between the main characteristics of the input data presented to them [22,51,87].…”

Section: Self-organising Mapsmentioning

confidence: 99%

“…As a result, inputs presented to MLP need to be of fixed length. Apart from this, MLPs are only able to deal with small vocabularies, making them more appropriate to phoneme recognition rather than word recognition [1, 2, 83].…”

Section: Automatic Speech Recognition Systemsmentioning

confidence: 99%

See 2 more Smart Citations

Comparative study of automatic speech recognition techniques

Cutajar

Gatt

Grech

et al. 2013

IET signal process.

View full text Add to dashboard Cite

Over the past decades, extensive research has been carried out on various possible implementations of automatic speech recognition (ASR) systems. The most renowned algorithms in the field of ASR are the mel-frequency cepstral coefficients and the hidden Markov models. However, there are also other methods, such as wavelet-based transforms, artificial neural networks and support vector machines, which are becoming more popular. This review article presents a comparative study on different approaches that were proposed for the task of ASR, and which are widely used nowadays. † training time increases linearly with increase in vocabulary size [42] † quantisation error in the discrete representation of speech signals [42] † temporal information is ignored [42] PCA † reduction in the feature vector's size, while retaining much of the significant information [131] † robust [59, 60] † computationally expensive for high-dimensional data [8] LDA † maximises the distance between classes, but minimises the within class distance [132] † robust [133] † sample distribution is assumed a priori to be Gaussian [63] † class samples are assumed to have equal variance [63] Classification technique Advantages Disadvantages HMM † able to model time distribution of speech signals [103] † simple to adapt [68] † capable to model a sequence of discrete or continuous symbols [13] † inputs can be of variable length [40] † based on the assumption that the probability of being in a particular state is dependent only on its preceding state, ignoring any long-term dependencies [82] † emission probabilities are arbitrarily chosen; hence, these might not even represent properly the output probabilities of the corresponding state [82] ANN (in general) † good classifiers [16, 45] † highly adequate for pattern recognition applications [16, 45] † self-organising [16, 45] † self-learning [16, 45] † self-adaptive in new environments [16, 45] † robust [7] † based on ERM; hence, prone to over training a local minima problems [45, 103] MLP † good discriminating ability [2] † unable to model time distribution of speech signals [2] † inputs have to be of fixed length [2] † able to deal with small vocabularies only [2] SOM † no a priori information is required for training a SOM [134] † can easily adapt if a new sample is presented to it [134] † capable of parallel computation [134] † SOM algorithm is not well defined mathematically; hence, values for the network parameters need to be found by trial-and-error [134] † ordered mapping obtained after the training phase may be lost when applied in real environments due to frequent adaptations [134] RBF † simple to implement [135] † Good discriminating ability [135] † robust [135] † online learning ability [135] † shift invariant in time [91] RNN † able to model time distribution of speech signals thanks to the feedback connections [95, 103] † complex training algorithm [94] † training algorithm is highly sensitive to any changes [94] FNN † does not need large amount of samples during the learning process [99] † ...

show abstract

Section: Multilayer Perceptronsmentioning

confidence: 99%

“…Hence, the Fig. 4 Multilayer perceptron [83] SOMs are capable of distinguishing between the main characteristics of the input data presented to them [22,51,87].…”

Section: Self-organising Mapsmentioning

confidence: 99%

See 1 more Smart Citation

Comparative study of automatic speech recognition techniques

Cutajar

Gatt

Grech

et al. 2013

IET signal process.

View full text Add to dashboard Cite

show abstract

“…This approach was successfully extended as two or three MLPs [8] and a MLP with a CRF [10]. MLP in the first level was also extended to have a sparse hidden layer in [13]. Tandem classifier approach was also called hierarchical phoneme posterior probability estimator in [9].…”

Section: Motivation Of Tandem Dbn Approachmentioning

confidence: 99%

Investigation of tandem deep belief network approach for phoneme recognition

Zheng

Shen

et al. 2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

This paper proposes using tandem DBN approach-a hierarchical architecture that consists of two or more deep belief networks (DBNs) in tandem manner-for phoneme recognition task on TIMIT. First we describe the standard DBN approach applied in phoneme recognition and discuss the motivation of combining it with tandem classifier approach. We then perform series of experiments to find out the best configuration for the DBN in the second level and discover the full potential of this method. The experiments show that for the DBN in the second level, (a) 2048 units in each hidden layer is better than 1024 and 512 units, (b) for sufficient length of temporal context, two hidden layers are better, (c) the one gives best performance on development set shows 4% relative improvement on coretest set.

show abstract

“…Hence, the mel frequency scale corresponds to a linear scale below 1 kHz, and algorithmic scale above the 1 kHz. [17].…”

Section: Fig 2 Stages Of Linear Predictive Coding B Mel Frequency Cmentioning

confidence: 99%

Speech Feature Extraction and Classification: A Comparative Review

Madan¹,

Gupta²

2014

IJCA

View full text Add to dashboard Cite

This paper gives a brief survey on speech recognition and presents an overview for various techniques used at various stages of speech recognition systems. Researchers has been working in this research area for many years however accuracy for speech recognition still attention for variation of context, speaker's variability, environment conditions .The development of speech recognition system requires certain concepts to be included-Defining different classes of speech, techniques for speech feature extraction, speech classification modeling and measuring system performance .The main aim of this paper is to discuss and compare different approaches used for feature extraction and classification stages in speech recognition system.

show abstract

Multilayer perceptron with sparse hidden outputs for phoneme recognition

Cited by 19 publications

References 9 publications

Comparative study of automatic speech recognition techniques

Comparative study of automatic speech recognition techniques

Investigation of tandem deep belief network approach for phoneme recognition

Speech Feature Extraction and Classification: A Comparative Review

Contact Info

Product

Resources

About