Multitree Genetic Programming With New Operators for Transfer Learning in Symbolic Regression With Incomplete Data

Al-Helali, Baligh; Chen, Qi; Xue, Bing; Zhang, Mengjie

doi:10.1109/tevc.2021.3079843

Cited by 20 publications

(19 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each parent can be subject to up to three different genetic operators: (i) crossover, (ii) subtree mutation, and (iii) one-point mutation, with probabilities of 𝑝 𝑐 , 𝑝 𝑠 , and 𝑝 𝑜 , respectively. These can be easily experimented with, using our publicly available code 1 . Note that the genetic operators, maximum tree size, and tree depth are applied to single trees within a multi-tree.…”

Section: Genetic Programming Parametersmentioning

confidence: 99%

“…For the crossover operator, where subtrees are exchanged between two trees, one has to decide whether crosspollination between trees at different index from the two multi-tree individuals is allowed. In [19], the authors randomly select a tree from each individual and perform standard crossover between them (random-index crossover), while in [1], the authors only allow trees at the same position in the multi-tree to crossover with each other (same-index crossover). Here, we use same-index crossover when mixing trees from different multi-trees.…”

Section: Genetic Programming Parametersmentioning

confidence: 99%

“…Here, we use same-index crossover when mixing trees from different multi-trees. The reason we make this choice is that same-index crossover encourages trees at the same 1 https://github.com/pinouche/gp_dr index to specialize [1] and converge towards learning the same latent dimension of the lower-dimensional representation.…”

Section: Genetic Programming Parametersmentioning

confidence: 99%

See 2 more Smart Citations

On genetic programming representations and fitness functions for interpretable dimensionality reduction

Uriot¹,

Virgolin²,

Alderliesten³

et al. 2022

Preprint

View full text Add to dashboard Cite

Dimensionality reduction (DR) is an important technique for data exploration and knowledge discovery. However, most of the main DR methods are either linear (e.g., PCA), do not provide an explicit mapping between the original data and its lower-dimensional representation (e.g., MDS, t-SNE, isomap), or produce mappings that cannot be easily interpreted (e.g., kernel PCA, neural-based autoencoder). Recently, genetic programming (GP) has been used to evolve interpretable DR mappings in the form of symbolic expressions. There exists a number of ways in which GP can be used to this end and no study exists that performs a comparison. In this paper, we fill this gap by comparing existing GP methods as well as devising new ones. We evaluate our methods on several benchmark datasets based on predictive accuracy and on how well the original features can be reconstructed using the lower-dimensional representation only. Finally, we qualitatively assess the resulting expressions and their complexity. We find that various GP methods can be competitive with state-of-the-art DR algorithms and that they have the potential to produce interpretable DR mappings. CCS CONCEPTS• Computing methodologies → Genetic programming; Dimensionality reduction and manifold learning.

show abstract

Section: Genetic Programming Parametersmentioning

confidence: 99%

Section: Genetic Programming Parametersmentioning

confidence: 99%

Section: Genetic Programming Parametersmentioning

confidence: 99%

See 1 more Smart Citation

On genetic programming representations and fitness functions for interpretable dimensionality reduction

Uriot¹,

Virgolin²,

Alderliesten³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…. Al-helali et al [4] proposed a multi-tree genetic programming method with new genetic operators based on transferred knowledge and utilised it for solving symbolic regression problems with incomplete data. Mu ñoz et al [171] have given a comprehensive review of transfer optimisation methods for GP in the context of constructive induction.…”

Section: Transfer Optimisation Methodsmentioning

confidence: 99%

“…Since the p-value indicates the existence of significant difference, the Nemenyi post-hoc comparison is performed and the obtained p-values are given in Table 5. 4.…”

Section: Effectiveness Of Guided Searchmentioning

confidence: 99%

Transfer Optimisation in Genetic Programming for Solving Uncertain Capacitated Arc Routing Problem

Ardeh¹

View full text Add to dashboard Cite

Uncertain Capacitated Arc Routing Problem (UCARP) is a combinatorial optimisation problem with many important real-world applications. Genetic programming (GP) is a powerful machine learning technique that has been successfully used to automatically evolve routing policies for UCARP. Reusability is an open issue in the field of UCARP and in this direction, an open challenge is the case of scenario changes, e.g. change in the number of vehicles and probability distributions of random demands, which typically requires new training procedures to be initiated. Considering the expensive training cost of evolving routing policies for UCARP, a promising strategy is to learn and reuse knowledge from a previous problem-solving process to improve the effectiveness and efficiency of solving a new related problem, i.e. transfer learning. The overall goal of this thesis is to develop novel knowledge transfer algorithms for GP for solving UCARP to handle environment changes more effectively and efficiently. To fulfil this goal, a plethora of machine learning techniques, i.e. surrogate models, feature selection, searching and specialised genetic operators, are utilised in this thesis. First, this thesis explores the effectiveness of the existing transfer optimisation methods for solving UCARP. Accordingly, one of the main directions of this thesis is towards identifying the nature of transferable knowledge, which can impact the quality of knowledge transfer for GP to solve UCARP. For this purpose, a collection of the state-of-the-art transfer optimisation GP algorithms are evaluated for UCARP. After identifying some potential gaps in the literature, a number of preliminary transfer optimisation algorithms are proposed that supplement the literature. To evaluate the algorithms, a large set of knowledge transfer scenarios with various source and target problems were designed based on real-world datasets. According to the results, none of the methods showed significant improvement in the effectiveness of the trained UCARP routing policies. These results revealed the need for more effective transfer optimisation methods specifically designed for UCARP. Furthermore, our investigations revealed that the presence of duplicates in knowledge sources is one of the main challenges for effective transfer optimisation in solving UCARP. Second, we propose approaches to handling the presence of duplicates in the transferred knowledge. The first approach increases population diversity after knowledge transfer to counteract the loss of diversity that is introduced by the presence of duplicates in the transferred knowledge. In the second approach, the duplicates are removed from the transferred knowledge. Then, the transferred knowledge is utilised to create a diverse initial GP population of high-quality individuals. Both approaches are investigated through detailed experimental studies. The results indicate that, while the first approach did not perform better than GP with knowledge transfer, the second can improve the effectiveness of training routing policies with GP significantly. Third, this thesis proposes a novel algorithm that transfers the phenotypic characteristics of the routing policies for solving the source problem. In the new algorithm, the most fit and unique source routing policies are utilised for initialising GP for solving the target problem. Then, a tabu list is placed on the source routing policies and the GP process is prohibited from recreating any of the source routing policies. The motivation for this approach is that, due to the existence of similarity between the source and target problems, source routing policies are unlikely to have a good performance for the target problem. Our experimental studies confirmed that by prohibiting GP from recreating source policies, and the computational resources will be spent on searching and evaluating new regions of the search space, which can lead to discovering better solutions. Fourth, this thesis proposes a novel knowledge transfer algorithm based on the idea of maintaining the transferred knowledge as an auxiliary population. In this approach, first, the best individuals of the duplicate-free knowledge source are used to initialise GP. Additionally, these transferred individuals are also maintained as an auxiliary population and are evolved alongside the main population. To save the computational cost, the auxiliary population is evolved with a surrogate method. Additionally, an elaborate knowledge exchange mechanism between the two populations is devised that emphasises transferring high-quality and unique individuals, the transfer of which can improve the diversity of the receiving population. This allows GP to overcome the problem of losing its population diversity during the evolutionary process. Our detailed experimental results confirmed the superior performance of the proposed algorithm and confirmed that the proposed method improved the phenotypic diversity of GP population.

show abstract

Evolutionary Regression and Modelling

Chen,

Xue,

Browne

et al. 2023

Genetic and Evolutionary Computation

View full text Add to dashboard Cite

Multitree Genetic Programming With New Operators for Transfer Learning in Symbolic Regression With Incomplete Data

Cited by 20 publications

References 28 publications

On genetic programming representations and fitness functions for interpretable dimensionality reduction

On genetic programming representations and fitness functions for interpretable dimensionality reduction

Transfer Optimisation in Genetic Programming for Solving Uncertain Capacitated Arc Routing Problem

Evolutionary Regression and Modelling

Contact Info

Product

Resources

About