Fast learning of relational kernels

Landwehr, Niels; Passerini, Andrea; Raedt, Luc De

doi:10.1007/s10994-009-5163-1

Cited by 57 publications

(68 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In most cases, these methods aim to develop a suitable representation of structured data for subsequent learning, not to discover features. There are previous works, however, where the feature space itself is learned from relational data [23,29] and is interpretable in terms of definite clauses.…”

Section: Resultsmentioning

confidence: 99%

Type Extension Trees for feature construction and learning in relational domains

Jaeger

Lippi²,

Passerini

2013

Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

Please cite this article in press as: M. Jaeger et al., Type extension trees for feature construction and learning in relational domains, Artificial Intelligence (2013), http://dx.doi.org/10. 1016/j.artint.2013.08.002 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. Type Extension Trees for Feature Construction and Learning in Relational DomainsManfred Jaeger AbstractType Extension Trees are a powerful representation language for "count-ofcount" features characterizing the combinatorial structure of neighborhoods of entities in relational domains. In this paper we present a learning algorithm for Type Extension Trees (TET) that discovers informative count-of-count features in the supervised learning setting. Experiments on bibliographic data show that TET-learning is able to discover the count-of-count feature underlying the definition of the h-index, and the inverse document frequency feature commonly used in information retrieval. We also introduce a metric on TET feature values. This metric is defined as a recursive application of the Wasserstein-Kantorovich metric. Experiments with a k-NN classifier show that exploiting the recursive count-of-count statistics encoded in TET values improves classification accuracy over alternative methods based on

show abstract

Section: Resultsmentioning

confidence: 99%

Type Extension Trees for feature construction and learning in relational domains

Jaeger

Lippi²,

Passerini

2013

Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

show abstract

“…The model, referred to as kFOIL, implements a dynamic propositionalization approach and allows one to perform both classification and regression tasks. In (Landwehr et al 2010), a general theoretical framework for statistical logical learning with kernels based on dynamic propositionalization is developed where structure learning corresponds to inferring a suitable kernel on logical objects, and parameter learning corresponds to function learning in the resulting reproducing kernel Hilbert space.…”

Section: Related Workmentioning

confidence: 99%

Bridging logic and kernel machines

et al. 2011

View full text Add to dashboard Cite

We propose a general framework to incorporate first-order logic (FOL) clauses, that are thought of as an abstract and partial representation of the environment, into kernel machines that learn within a semi-supervised scheme. We rely on a multi-task learning scheme where each task is associated with a unary predicate defined on the feature space, while higher level abstract representations consist of FOL clauses made of those predicates. We re-use the kernel machine mathematical apparatus to solve the problem as primal optimization of a function composed of the loss on the supervised examples, the regularization term, and a penalty term deriving from forcing real-valued constraints deriving from the predicates. Unlike for classic kernel machines, however, depending on the logic clauses, the overall function to be optimized is not convex anymore. An important contribution is to show that while tackling the optimization by classic numerical schemes is likely to be hopeless, a stage-based learning scheme, in which we start learning the supervised examples until convergence is reached, and then continue by forcing the logic clauses is a viable direction to attack the problem. Some promising experimental results are given on artificial learning tasks and on the automatic tagging of bibtex entries to emphasize the comparison with plain kernel machines.

show abstract

“…This means that using ILP systems just based on default values for parametersthe accepted practice at present-can give misleading estimates of the best response possible from the system. This is illustrated in Figure 13, which shows estimated accuracies on other data sets reported in the literature that also use the Aleph system with default values for all parameters (these data sets have been used widely: see, for example, Landwehr et al, 2006 andMuggleton et al, 2008). Taken with our previous results for the mutagenesis and carcinogenesis data (we will only use the B max results, as these are the results used in the literature), we are now able to make some statements of statistical significance.…”

Section: Resultsmentioning

confidence: 99%

Parameter Screening and Optimisation for ILP Using Designed Experiments

Srinivasan

Ramakrishnan

2010

Inductive Logic Programming

View full text Add to dashboard Cite

Reports of experiments conducted with an Inductive Logic Programming system rarely describe how specific values of parameters of the system are arrived at when constructing models. Usually, no attempt is made to identify sensitive parameters, and those that are used are often given "factory-supplied" default values, or values obtained from some non-systematic exploratory analysis. The immediate consequence of this is, of course, that it is not clear if better models could have been obtained if some form of parameter selection and optimisation had been performed. Questions follow inevitably on the experiments themselves: specifically, are all algorithms being treated fairly, and is the exploratory phase sufficiently well-defined to allow the experiments to be replicated? In this paper, we investigate the use of parameter selection and optimisation techniques grouped under the study of experimental design. Screening and response surface methods determine, in turn, sensitive parameters and good values for these parameters. Screening is done here by constructing a stepwise regression model relating the utility of an ILP system's hypothesis to its input parameters, using systematic combinations of values of input parameters (technically speaking, we use a two-level fractional factorial design of the input parameters). The parameters used by the regression model are taken to be the sensitive parameters for the system for that application. We then seek an assignment of values to these sensitive parameters that maximise the utility of the ILP model. This is done using the technique of constructing a local "response surface". The parameters are then changed following the path of steepest ascent until a locally optimal value is reached. This combined use of parameter selection and response surface-driven optimisation has a long history of application in industrial engineering, and its role in ILP is demonstrated using well-known benchmarks. The results suggest that computational overheads from this preliminary phase are not substantial, and that much can be gained, both on improving system performance and on enabling controlled experimentation, by adopting well-established procedures such as the ones proposed here.Keywords: inductive logic programming, parameter screening and optimisation, experimental design * . A.S. also holds an adjunct position at the School of CSE, University of New South Wales, Sydney; and visiting position at the Oxford University Computing Laboratory, Oxford.

show abstract

Fast learning of relational kernels

Cited by 57 publications

References 55 publications

Type Extension Trees for feature construction and learning in relational domains

Type Extension Trees for feature construction and learning in relational domains

Bridging logic and kernel machines

Parameter Screening and Optimisation for ILP Using Designed Experiments

Contact Info

Product

Resources

About