Aportaciones al modelado conexionista de lenguaje y su aplicación al reconocimiento de secuencias y traducción automática

Martínez, Zamora; Julián, Francisco Javier Goicolea

doi:10.4995/thesis/10251/18066

Cited by 1 publication

(12 citation statements)

References 114 publications

(224 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…NN LMs [Bengio et al 2003;Castro et al 2001a;Castro and Prat 2003;Schwenk 2007;Zamora-Martínez 2012] are easy to understand in this ANNs learn posteriors context since they just take profit of the general capability of artificial neural networks (ANNs) to estimate posterior probabilities. In order to estimate p(w|h), a suitable representation of h is required to feed the network inputs and a way to classify among the different words of the vocabulary Ω is also required.…”

Section: Neural Network N-grams (Nnlms)mentioning

confidence: 99%

“…• use the shortlist approach at the input layer. The use of the shortlist approach at the output layer was described in Section 6.3.3 and is widely reported in the literature, but it can also be used at the input layer as described in [Zamora-Martínez 2012];…”

Section: Nn-mentioning

confidence: 99%

“…• memorization of softmax normalization constants to avoid the computation of the entire output layer on trained models Zamora-Martínez 2012], an improvement over this basic idea, based on training skipping NN LMs, is described in this section;…”

Section: Nn-mentioning

confidence: 99%

“…This section proposes an improvement over the memoization of softmax normalization constants 55 described in Zamora-Martínez 2012]. Two different alternatives exist depending on what to do when a normalization constant has not been pre-computed: to compute it on-the-fly 56 or to construct what has been coined a "Fall Back NN LMs" as follows:…”

Section: Fast Evaluation With Skipping Nn Lmsmentioning

confidence: 99%

“…Note that, although this procedure can also be applied to the original fast evaluation approach from Zamora-Martínez 2012], what makes skipping NN LMs more suitable for this pruning technique is that the same MLP is being used in all the cases. This brings the possibility of incremental computing the different input combinations and the possibility of measuring the influence of certain inputs by computing the error gradient, or by directly comparing the activations of the last hidden layer.…”

Section: Fast Evaluation With Skipping Nn Lmsmentioning

confidence: 99%

See 4 more Smart Citations

Contributions to the joint segmentation and classification of sequences (My two cents on decoding and handwriting recognition)

Boquera¹

View full text Add to dashboard Cite

To my family, past, present and future, very specially to my parents and to my wife Manuela for their unconditional love and support. v 65 3.1.8 Prediction/Forecasting 65 3.1.9 Translation 65 3.1.10 Alignment 66 3.1.11 KWS, indexing, querying by example 69 3.1.12 Unsupervised term discovery 72 3.1.13 Filtering/Denoising and Smoothing 73 3.1.14 Summarization 73 3.1.15 Joint Segmentation and Classification 74 3.2 Some extensions 80 3.2.1 Several input sequences 80 3.2.2 Interactive systems 82 xv xvi Contents 3.3 Limitations, how to face them, new proposal 87 3.4 Questionnaire when faced with a new problem 91 3.5 Analysis and evaluation measures 93 3.5.1 Quality assessment 96 3.5.2 Performance 99 3.5.3 Interaction 100 3.5.4 Other measures 102 3.6 Summary and some conclusions 102 105 4.1 Some preliminary ML concepts 106 4.1.1 (Probabilistic) graphical models 115 4.2 Two stage generative model 130 4.2.1 Hierarchy for the second stage 133 4.2.2 Limitations, extensions and generalizations 137 4.3 Classical problems of two stage generative models 141 4.3.1 Probability of observation 142 4.3.2 Decoding 144 4.3.3 Model estimation 145 4.4 Some alternative models 146 4.4.1 Relationship with Dynamic Graphical Models 146 4.4.2 Fixed dimension feature segments 150 4.4.3 Estimation of frame-wise segment posteriors 155 4.4.4 Graph transformer networks 160 4.4.5 Some non-probabilistic frameworks 163 4.5 Summary and some conclusions 165 , , , 167 5.1 Introduction 167 5.2 Recognition, parsing, decoding 170 5.3 Weighted languages and semirings 172 5.4 Some formalisms 177 5.4.1 Formal/generative grammars 177 5.4.2 Finite state automata and transducers 184 5.4.3 Recurrent transition networks 187 5.5 Deriving the composition of a regular and a CF model 194 5.5.1 State-pair transducer composition 195 5.5.2 Extension to null-transitions 200 5.5.3 Extension to model reference transitions 209 5.5.4 Transformation to homogeneous epsilon form 220 5.6 Review of parsing approaches, decoders and algorithms 225 5.7 From composition to recognition/decoding 232 5.7.1 Acyclic inputs 233 5.7.2 Semiring specific optimizations 235 5.8 Summary and some conclusions 235 239 6.1 Probabilistic decompositions 242 6.1.1 Chain rule, clustering histories 243 6.1.2 Whole sentence LMs 244 6.1.3 Combining spans of the sequence 245 6

show abstract