Active learning differs from "learning from examples" in that the learning algorithm assumes at least some control over what part of the input domain it receives information about. In some situations, active learning is provably more powerful than learning from examples alone, giving better generalization for a fixed number of training examples. In this article, we consider the problem of learning a binary concept in the absence of noise. We describe a formalism for active concept learning called selective sampling and show how it may be approximately implemented by a neural network. In selective sampling, a learner receives distribution information from the environment and queries an oracle on parts of the domain it considers "useful." We test our implementation, called an SGnetwork, on three domains and observe significant improvement in generalization.

For many t ypes of machine learning algorithms, one can compute the statistically optimal" way to select training data. In this paper, we review how optimal data selection techniques have been used with feedforward neural networks. We then show h o w the same principles may be used to select data for two alternative, statistically-based learning architectures: mixtures of Gaussians and locally weighted regression. While the techniques for neural networks are computationally expensive and approximate, the techniques for mixtures of Gaussians and locally weighted regression are both e cient and accurate. Empirically, w e observe that the optimality criterion sharply decreases the number of training examples the learner needs in order to achieve good performance.

Public reporting burden for this collection of infornation is estimated to average 1 hour per response. Including the time for revtewing instructions searching existing data sourcoes•gathering and maintaining the data needed, and comnpleting and reviewing the collection of rnfornration. Send comments regarding this burden estimate or any other aspect oil this collect ion of informnationĩ nciuding suggestions for reducing this burden to Washington Headquarters Services, Directorate for Information Operations and Reports,

I consider the question "How should one act when the only goal is to learn as much as possible?". Building on the theoretical results of Fedorov (1972, Theory of Optimal Experiments, Academic Press) and MacKay (1992, Neural Computation, 4, 590-604), I apply techniques from optimal experiment design (OED) to guide the query/action selection of a neural network learner. I demonstrate that these techniques allow the learner to minimize its generalization error by exploring its domain efficiently and completely. I conclude that, while not a panacea, OED-based query/action selection has much to offer, especially in domains where its high computational costs can be tolerated. Copyright 1996 Elsevier Science Ltd