Automatic string replace by examples

Abstract

Search-and-replace is a text processing task which may be largely automated with regular expressions: the user must describe with a specific formal language the regions to be modified (search pattern) and the corresponding desired changes (replacement expression). Writing and tuning the required expressions requires high familiarity with the corresponding formalism and is typically a lengthy, error-prone process.In this paper we propose a tool based on Genetic Programming (GP) for generating automatically both the search pattern and the replacement expression based only on examples. The user merely provides examples of the input text along with the desired output text and does not need any knowledge about the regular expression formalism nor about GP. We are not aware of any similar proposal. We experimentally evaluated our proposal on 4 different searchand-replace tasks operating on real-world datasets and found good results, which suggests that the approach may indeed be practically viable.

Automatic Search-and-Replace From Examples With Coevolutionary Genetic Programming

“…Our work constitutes a significant improvement over our earlier proposal [5], which, to the best of our knowledge, was the first system able to generate automatically both the search pattern and the replacement expression based only on examples. Our earlier proposal was based on GP and consisted of three steps: (i) a first evolutionary search aimed at constructing a preliminary search pattern; (ii) construction of a replacement expression of predefined structure, tailored to the provided examples; and, (iii) a further evolutionary search aimed at constructing the search pattern that, coupled with the replacement expression obtained at the previous step, would solve the user-specified task.…”

“…In particular, we introduced the enforcement of a phenotypic diversity promotion scheme [7], we extended the number and variety of individuals in the initial population that are generated based on the available examples (rather than at random) and, most importantly, we introduced a significantly different fitness function. In our earlier work [5], we used a multi-objective fitness function based on two indexes: the amount of characterlevel errors and a quantification of the structural mismatch between the two components of a candidate solution, i.e., search pattern and replacement expression (please refer to the cited paper for full details). In this work we introduce a new fitness function composed of three objectives: the amount of character-level errors, a quantification of the ability to identify all the substrings in the examples that have to be modified (a form of character-level recall), a complexity measure of the candidate solution.…”

“…We assessed our new proposal on six tasks of realistic complexity, four of which were included also in [5], and considered two significant baselines: our earlier GP proposal [5] and the FlashFill algorithm incorporated into Microsoft Excel [8].…”

Learning Text Patterns Using Separate-and-Conquer Genetic Programming

“…Our pattern evolutionary search is built upon the approaches proposed in [2,12,18], which we extend in three key aspects: (i) different fitness definitions (we use three objectives rather than two objectives); (ii) different fitness comparison criteria (we use a hierarchy between the fitness indexes rather than a Paretoranking); and, (iii) a mechanism for enforcing diversity among individuals.…”

“…, 9; (iv) partial ranges obtained from the slices in (s,X d ,Xu)∈T X d according to the procedure described in [12]-a partial range being the largest interval of characters occurring in a set of strings (e.g., a-c and l-n are two partial ranges obtained from {cabin, male}), see the cited paper for full details); (v) other special characters such as \., :, @, and so on. The initialization of the population of n pop individuals is based on the slices in (s,X d ,Xu)∈T X d , as follows (similarly to [18]). For each slice x s ∈ (s,X d ,Xu)∈T X d , two individuals are built: one whose string representation is equal to the content of x s where each digit is replaced by \d and each other alphabetic character is replaced by \w; another individual whose string representation is the same as the former and where consecutive occurrences of \d (or \w) are replaced by \d++ (or \w++).…”