A Comparative Study of ID3 and Backpropagation for English Text-to-Speech Mapping

Dietterich, Thomas G.; Hild, Hermann; Bakiri, Ghulum

doi:10.1016/b978-1-55860-141-3.50007-9

Cited by 49 publications

(30 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A noteworthy implementation of such a decomposition is the recent DNN 'Uber-Net' (Kokkinos, 2016), which solves 7 vision related tasks (boundary, surface normals, saliency, semantic segmentation, semantic boundary and human parts detection) with a single multi-scale DNN network to reduce the memory footprint. It can be assumed that such a multi-task training improves convergence speed and better generalization to unseen data, something that already has been observed on other multi-task setups related to speech processing, vision and maze navigation (Bilen & Vedaldi, 2016;Caruana, 1998;Dietterich, Hild, & Bakiri, 1990, 1995Mirowski et al, 2016).…”

Section: Box 1 Deep Neural Networkmentioning

confidence: 77%

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Scholte

Losch

Ramakrishnan

et al. 2018

Cortex

View full text Add to dashboard Cite

Deep learning Cost functions RepresentationsVisual processing a b s t r a c t Vision research has been shaped by the seminal insight that we can understand the higher-tier visual cortex from the perspective of multiple functional pathways with different goals. In this paper, we try to give a computational account of the functional organization of this system by reasoning from the perspective of multi-task deep neural networks. Machine learning has shown that tasks become easier to solve when they are decomposed into subtasks with their own cost function. We hypothesize that the visual system optimizes multiple cost functions of unrelated tasks and this causes the emergence of a ventral pathway dedicated to vision for perception, and a dorsal pathway dedicated to vision for action.To evaluate the functional organization in multi-task deep neural networks, we propose a method that measures the contribution of a unit towards each task, applying it to two networks that have been trained on either two related or two unrelated tasks, using an identical stimulus set. Results show that the network trained on the unrelated tasks shows a decreasing degree of feature representation sharing towards higher-tier layers while the network trained on related tasks uniformly shows high degree of sharing.We conjecture that the method we propose can be used to analyze the anatomical and functional organization of the visual system and beyond. We predict that the degree to which tasks are related is a good descriptor of the degree to which they share downstream cortical-units.

show abstract

Section: Box 1 Deep Neural Networkmentioning

confidence: 77%

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Scholte

Losch

Ramakrishnan

et al. 2018

Cortex

View full text Add to dashboard Cite

show abstract

“…Both symbolic and artificial neural network (or connectionist) learning algorithms have been developed; however, until recently Mooney, Shavlik, Towell, & Gove, 1989;Weiss & Kapouleas, 1989;Atlas, et al, 1990;Dietterich, Hild, & Bakiri, 1990) there has been little direct comparison of these two basic approaches to machine learning. Consequently, despite the fact that symbolic and connectionist learning systems frequently address the same general problem, very little is known regarding their comparative strengths and weaknesses.…”

Section: Introductionmentioning

confidence: 99%

Symbolic and neural learning algorithms: An experimental comparison

1991

View full text Add to dashboard Cite

Abstract. Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perceptron and backpropagation neural learning algorithms have been performed using five large, real-world data sets, Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs. Backpropagation occasionally outperforms the other two systems when given relatively small amounts of training data. It is slightly more accurate than ID3 when examples are noisy or incompletely specified. Finally, backpropagation more effectively utilizes a "distributed" output encoding.

show abstract

“…These domains have received considerable attention from connectionist researchers who employed the back propagation learning algorithm (Sejnowski & Rosenberg, 1986;Qian & Sejnowski, 1988;Towell et al, 1990). In addition, the word pronunciation problem has been the subject of a number of comparisons using other machine learning algorithms (Stanfill & Waltz, 1986;Shavlik et al, 1989;Dietterich et al, 1990). All of these domains represent problems of considerable practical importance, and all have symbolic feature values, which makes them difficult for conventional nearest neighbor algorithms.…”

Section: Instance-based Learning Versus Other Modelsmentioning

confidence: 99%

A weighted nearest neighbor algorithm for learning with symbolic features

1993

View full text Add to dashboard Cite

Abstract. In the past, nearest neighbor algorithms for learning from examples have worked best in domains in which all features had numeric values. In such domains, the examples can be treated as points and distance metrics can use standard defioJtions. In symbolic domains, a more sophisticated treatment of the feature space is required. We introduce a nearest neighbor algorithm for learning in domains with symbolic features. Our algorithm calculates distance tables that allow it to produce real-valued distances between instances, and attaches weights to the instances to further modify the structure of feature space. We show that this technique produces excellent classification accuracy on three problems that have been studied by machine learning researchers: predicting protein secondary structure, identifying DNA promoter sequences, and pronouncing English text. Direct experimental comparisons with the other learning algorithms show that our nearest neighbor algorithm is comparable or superior in all three domains. In addition, our algorithm has advantages in training speed, simplicity, and perspicuity. We conclude that experimental evidence favors the use and continued development of nearest neighbor algorithms for domains such as the ones studied here.

show abstract

A Comparative Study of ID3 and Backpropagation for English Text-to-Speech Mapping

Cited by 49 publications

References 9 publications

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Visual pathways from the perspective of cost functions and multi-task deep neural networks

Symbolic and neural learning algorithms: An experimental comparison

A weighted nearest neighbor algorithm for learning with symbolic features

Contact Info

Product

Resources

About