No abstract

Unlike a univariate decision tree, a multivariate decision tree is not restricted to splits of the instance space that are orthogonal to the features' axes. This article addresses several issues for constructing multivariate decision trees: representing a multivariate test, including symbolic and numeric features, leaming the coefficients of a multivariate test, selecting the features to include in a test, and pruning of multivariate decision trees. We present several new methods for forming multivariate decision trees and compare them with several well-known methods. We compare the different methods across a variety of learning tasks, in order to assess eaeh method's ability to find concise, accurate decision trees. The results demonstrate that some multivariate rnethods are in general more effective than others (in the context of our experimental assumptions). In addition, the experiments confirm that allowing multivariate tests generally improves the accuracy of the resulting deeision tree over a univariate tree.

No abstract

This chapter concerns learning heuristic problem-solving strategies through experience. In particular, we focus on the issue of learning heuristics to guide a forward-search problem solver, and describe a computer program called LEX, which acquires problem-solving heuristics in the domain of symbolic integration. LEX acquires and modifies heuristics by iteratively applying the following process: (i) generate a practice problem; (ii) use available heuristics to solve this problem; (iii) analyze the search steps performed in obtaining the solution; and (iv) propose and refine new domain-specific heuristics to improve performance on subsequent problems. We describe the methods currently used by LEX, analyze strengths and weaknesses of these methods, and discuss our current research toward more powerful approaches to learning heuristics. INTRODUCTIONEfforts to build powerful, specialized, heuristic problem solvers have met with increasing success over the past decade. However, identifying and encoding the domain-specific heuristics necessary for high performance of these systems is a painstaking, difficult process. As the complexity of a heuristic program grows, it becomes increasingly difficult for the system builder to predict how the addition of a particular new heuristic or operator will affect overall system performance. In response to this problem, there has been increased interest over the past several years in developing semi-automated and fully-automated methods to help construct expert heuristic problem solvers [Waterman, 1970;Davis, 1981;Buchanan, 1978;Politakis, 1979] (See also Chapter 7 of this book). At the same time, in the Cognitive Psychology literature there have been several attempts to model acquisition of problem-solving skills in humans [Anzai, 1979;Neves, 1978] (See also Chapter 7 of this book).The research presented here is directed toward devising methods by which heuristic problem-solving programs improve their problem-solving expertise through experience, by generating selected problems in the domain, solving them, and learning by analyzing their solutions. As part of this research we have designed and constructed a computer program, called LEX, that incorporates general methods for discovering domain-dependent problem-solving heuristics.The organization of this chapter is as follows. The learning problem considered by LEX is described, followed by a discussion of the methods employed by the current system. This includes methods for (i) solving practice problems, (ii) performing the credit assignment task of isolating appropriate and inappropriate search steps, (iii) proposing and generalizing heuristics, and (iv) generating new practice problems with which to experiment. The final sections of the chapter discuss augmenting the system by giving it knowledge to conduct detailed analyses of problem solutions. This knowledge can be used to provide strong guidance for the generalization process, and to generate new terms in the language with which heuristics are described. Some of the materi...

This article presents a case study in examining the bias of two particular formalisms: decision trees and linear threshold units. The immediate result is a new hybrid representation, called a 'perceptron tree: and an associated learning algorithm called the 'percepton tree error correction procedure: The longer term result is a model for exploring issues related to understanding representational bias and consmcting other useful hybrid representations.