A fast and compact elimination method of empty elements from a double‐array structure

Oono, Masaki; Atlam, El‐Sayed; Fuketa, Masao; Morita, Ken; Aoe, Jun-ichi

doi:10.1002/spe.545

Cited by 11 publications

(5 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, their comparative evaluation with the list representation for Tries shows that their proposal is slower for the insertion operation, and concludes that their structure should be improved for large sets of keys. Several authors have tried to overcome these drawbacks, such as [43] who tried to speed up the insertions, [44] who tried to eliminate efficiently the empty elements, [45] who tried to compact the double array structure, and [46] who tried to divide the Trie into multiple levels and to remove the BASE array.…”

Section: Related Workmentioning

confidence: 99%

MergedTrie: Efficient textual indexing

Ferrández

Peral

2019

PLoS ONE

View full text Add to dashboard Cite

The accessing and processing of textual information (i.e. the storing and querying of a set of strings) is especially important for many current applications (e.g. information retrieval and social networks), especially when working in the fields of Big Data or IoT, which require the handling of very large string dictionaries. Typical data structures for textual indexing are Hash Tables and some variants of Tries such as the Double Trie (DT). In this paper, we propose an extension of the DT that we have called MergedTrie . It improves the DT compression by merging both Tries into a single and by segmenting the indexed term into two fixed length parts in order to balance the new Trie. Thus, a higher overlapping of both prefixes and suffixes is obtained. Moreover, we propose a new implementation of Tries that achieves better compression rates than the Double-Array representation usually chosen for implementing Tries. Our proposal also overcomes the limitation of static implementations that does not allow insertions and updates in their compact representations. Finally, our MergedTrie implementation experimentally improves the efficiency of the Hash Tables, the DTs, the Double-Array, the Crit-bit, the Directed Acyclic Word Graphs (DAWG), and the Acyclic Deterministic Finite Automata (ADFA) data structures, requiring less space than the original text to be indexed.

show abstract

Section: Related Workmentioning

confidence: 99%

MergedTrie: Efficient textual indexing

Ferrández

Peral

2019

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…This is because elimination is not possible if there are fewer empty elements than compression elements. Oono et al improved Morita's method by focusing on single nodes with no siblings. Oono's method applies empty elements and single elements to the relocation addresses.…”

Section: Da Trie Dictionariesmentioning

confidence: 99%

“…A DA structure can include empty elements. Although its space efficiency depends on the load factor (ie, the proportion of nonempty elements to the total elements), as in a hash table, 22 existing methods [23][24][25][26] can maintain a high load factor.…”

Section: Introductionmentioning

confidence: 99%

Practical rearrangement methods for dynamic double‐array dictionaries

et al. 2017

Self Cite

View full text Add to dashboard Cite

SummaryDouble-array structures have been widely used to implement dictionaries with string keys. Although the space efficiency of dynamic double-array dictionaries tends to decrease with key updates, we can still maintain high efficiency using existing methods. However, these methods have practical problems of time and functionality. This paper presents several efficient rearrangement methods to solve these problems.Through experiments using real-world datasets, we demonstrate that the proposed rearrangement methods are much more practical than existing methods.

show abstract

“…13 To achieve a higher performance in DAACs, it is essential to carefully design the data structure with regard to the target applications and data characteristics. In the three decades since the original idea of the double-array was proposed by Aoe, 4 many implementation techniques have been developed from different points of view, such as scalability, 7,14,15 cache efficiency, 16,17 construction speed, [18][19][20] and specializations. 10,11,20 In addition to these academic studies, many open-source libraries have been developed, for example, References 5,6,21, and 22, which sometimes contain original techniques.…”

Section: Introductionmentioning

confidence: 99%

Engineering faster double‐array Aho–Corasick automata

Kanda¹,

Akabe²,

Oda

2023

Softw Pract Exp

View full text Add to dashboard Cite

Multiple pattern matching in strings is a fundamental problem in text processing applications such as regular expressions or tokenization. This article studies efficient implementations of double-array Aho-Corasick automata (DAACs), data structures for quickly performing the multiple pattern matching. The practical performance of DAACs is improved by carefully designing the data structure, and many implementation techniques have been proposed thus far. A problem in DAACs is that comprehensive descriptions and experimental analyses on their ideas are not provided. Engineers face difficulties in implementing an efficient DAAC. In this article, we review implementation techniques for DAACs and provide a comprehensive description of them. We also propose several new techniques for further improvement. We conduct exhaustive experiments through real-world datasets and reveal the best combination of techniques to achieve a higher performance in DAACs. The best combination is different from those used in the most popular libraries of DAACs, which demonstrates that their performance can be further enhanced. On the basis of our experimental analysis, we developed a new Rust library for fast multiple pattern matching using DAACs, named Daachorse, as open-source software at https://github. com/daac-tools/daachorse. Experiments demonstrate that Daachorse outperforms other AC-automaton implementations, indicating its suitability as a fast alternative for multiple pattern matching in many applications.

show abstract

A fast and compact elimination method of empty elements from a double‐array structure

Cited by 11 publications

References 5 publications

MergedTrie: Efficient textual indexing

MergedTrie: Efficient textual indexing

Practical rearrangement methods for dynamic double‐array dictionaries

Engineering faster double‐array Aho–Corasick automata

Contact Info

Product

Resources

About