Towards an evolutionary-based approach for natural language processing

Manzoni, Luca; Jakobović, Domagoj; Mariot, Luca; Picek, Stjepan; Castelli, Mauro

doi:10.1145/3377930.3390248

Cited by 5 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A number of works have investigated the combination of evolutionary computation and embedding representations. In [25,37,38], word embedding representations learned by means of word2vec are used as inputs to evolve genetic programs which are able to solve different natural language processing tasks. The dimension of the embeddings and the choice of the genetic programming fitness function were shown to play an important role in algorithm performance.…”

Section: Evolutionary Algorithms and Embedding Representationsmentioning

confidence: 99%

Boomerang-shaped neural embeddings for NK landscapes

Santana

Liefooghe

Derbel

2022

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

Understanding the landscape underlying NK models is of fundamental interest. Different representations have been proposed to better understand how the ruggedness of the landscape is influenced by the model parameters, such as the problem dimension, the degree of non-linearity and the structure of variable interactions. In this paper, we propose to use neural embedding, that is a continuous vectorial representation obtained as a result of applying a neural network to a prediction task, in order to investigate the characteristics of NK landscapes. The main assumption is that neural embeddings are able to capture important features that reflect the difficulty of the landscape. We propose a method for constructing NK embeddings, together with metrics for evaluating to what extent this embedding space encodes valuable information from the original NK landscape. Furthermore, we study how the embedding dimensionality and the parameters of the NK model influence the characteristics of the NK embedding space. Finally, we evaluate the performance of optimizers that solve the continuous representations of NK models by searching for solutions in the embedding space. CCS CONCEPTS• Theory of computation → Optimization with randomized search heuristics; • Computing methodologies → Discrete space search; Neural networks.

show abstract

Section: Evolutionary Algorithms and Embedding Representationsmentioning

confidence: 99%

Boomerang-shaped neural embeddings for NK landscapes

Santana

Liefooghe

Derbel

2022

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

show abstract

“…In Manzoni et al [6], the authors describe a process by which they use word embeddings instead of the words themselves as input to the genetic algorithm. They first mapped every word of sample sentences of k length to a wor2vec vector, applying mathematical operations to the vectors and decoding the modified vectors, finally interpreting them as words.…”

Section: Previous Genetic Algorithm Approaches To Text Generationmentioning

confidence: 99%

“…Although researchers have used genetic algorithms for text generation [6] [5], genetic algorithms have received little attention relative to deep learning approaches. In this paper, we propose an adversarial approach to the problem of text generation with genetic algorithms.…”

Section: Introductionmentioning

confidence: 99%

Towards Adversarial Genetic Text Generation

Kavi¹

2021

Computer Science &Amp; Information Technology (CS &Amp; IT)

View full text Add to dashboard Cite

Text generation is the task of generating natural language, and producing outputs similar to or better than human texts. Due to deep learning’s recent success in the field of natural language processing, computer generated text has come closer to becoming indistinguishable to human writing. Genetic Algorithms have not been as popular in the field of text generation. We propose a genetic algorithm combined with text classification and clustering models which automatically grade the texts generated by the genetic algorithm. The genetic algorithm is given poorly generated texts from a Markov chain, these texts are then graded by a text classifier and a text clustering model. We then apply crossover to pairs of texts, with emphasis on those that received higher grades. Changes to the grading system and further improvements to the genetic algorithm are to be the focus of future research.

show abstract

“…To experimentally assess our method, we loosely followed the GP parameter settings that we adopted in [21] for another supervised learning task, namely next word prediction, and checked with preliminary experiments that they were suitable for the image inpainting task as well. In particular, in each GP run, we evolved a population of 500 individuals for 500 generations, which amounts to 250 000 evaluations.…”

Section: Common Parametersmentioning

confidence: 99%

“…Hence, more than comparing with state-of-theart deep learning methods such as CNNs and GANs (which we leave for future research), the main motivation of our work is to search for preliminary evidence that convolutional inpainting can also be performed with Genetic Programming as an underlying learning primitive. Incidentally, we adopted a similar approach in [21] for the domain of automatic text generation. For these reasons, we frame the investigation presented in this paper around two general research questions:…”

Section: Introductionmentioning

confidence: 99%

CoInGP

Jakobović

Manzoni

Mariot

et al. 2021

Proceedings of the Genetic and Evolutionary Computation Conference

Self Cite

View full text Add to dashboard Cite

We investigate the use of Genetic Programming (GP) as a convolutional predictor for missing pixels in images. The training phase is performed by sweeping a sliding window over an image, where the pixels on the border represent the inputs of a GP tree. The output of the tree is taken as the predicted value for the central pixel. We consider two topologies for the sliding window, namely the Moore and the Von Neumann neighborhood. The best GP tree scoring the lowest prediction error over the training set is then used to predict the pixels in the test set. We experimentally assess our approach through two experiments. In the first one, we train a GP tree over a subset of 1000 complete images from the MNIST dataset. The results show that GP can learn the distribution of the pixels with respect to a simple baseline predictor, with no significant differences observed between the two neighborhoods. In the second experiment, we train a GP convolutional predictor on two degraded images, removing around 20% of their pixels. In this case, we observe that the Moore neighborhood works better, although the Von Neumann neighborhood allows for a larger training set. CCS CONCEPTS• Computing methodologies → Image processing; Genetic programming; Bio-inspired approaches.

show abstract

Towards an evolutionary-based approach for natural language processing

Cited by 5 publications

References 22 publications

Boomerang-shaped neural embeddings for NK landscapes

Boomerang-shaped neural embeddings for NK landscapes

Towards Adversarial Genetic Text Generation

CoInGP

Contact Info

Product

Resources

About