Graphs cannot be indexed in polynomial time for sub-quadratic time string matching, unless SETH fails

Equi, Massimo; Mäkinen, Veli; Tomescu, Alexandru I.

doi:10.48550/arxiv.2002.00629

Cited by 4 publications

(15 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, let 0 < ϵ < 1 be a constant and A be an NFA of size m and width at most p ≤ (ϵ/2) log 2 m. Contributions (2) and ( 4) -when combined with the polynomial-time DFA indexing algorithms of [5] imply that in polynomial time we can index A so that pattern matching queries on L(A) are solved in O(πm ϵ ) time. This breaks asymptotically the conditional lower bound Ω(πm) of Equi et al [9] holding in the worst-case even when polynomial preprocessing time is allowed.…”

Section: Introductionmentioning

confidence: 78%

“…An important consequence of the above bounds is that, in polynomial time, we can index an interesting subset of the regular languages (represented as NFAs) such that pattern matching query times break the indexability lower bound of Equi et al [9] (holding in the worst-case even when polynomial preprocessing time is allowed): ▶ Corollary 13. Let A be an NFA with m edges and width(A) ≤ (ϵ/2) log 2 m for any constant 0 < ϵ < 1.…”

Section: :8mentioning

confidence: 97%

“…All running times for building the index are polynomial in m and n, that is, in the size of the input. ◀ Before Corollary 13, only the case p = 1 (Wheeler languages [10,2]) admitted a polynomialtime indexing strategy (by the indexing algorithms of Alanko et al [1]) beating the indexability lower bound [9] of Ω(πm). Corollary 13 extends this result to a larger class of NFAs.…”

Section: :8mentioning

confidence: 99%

“…When L is represented as an NFA (equivalently, a regular expression) of size m, existing on-line algorithms [3] solve the problem in O(πm) time, π being the length of the query pattern. Recent lower bounds by Backurs and Indyk [4], Equi et al [8,7], Potechin and Shallit [16], and Gibney [11] show that, unless important conjectures such as the Strong Exponential Time Hypothesis (SETH) [12] fail, this complexity cannot be significantly improved. This holds even in the off-line setting (the subject matter of our work) where L can be pre-processed in an index in polynomial time and the complexity is measured in terms of query times [9].…”

Section: Introductionmentioning

confidence: 99%

“…Recent lower bounds by Backurs and Indyk [4], Equi et al [8,7], Potechin and Shallit [16], and Gibney [11] show that, unless important conjectures such as the Strong Exponential Time Hypothesis (SETH) [12] fail, this complexity cannot be significantly improved. This holds even in the off-line setting (the subject matter of our work) where L can be pre-processed in an index in polynomial time and the complexity is measured in terms of query times [9]. As pointed out by Backurs and Indyk [4], Gagie et al [10], Alanko et al [1,2], and Cotumaccio and Prezza [5], however, this does not necessarily mean that all regular languages are hard to index.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Which Regular Languages can be Efficiently Indexed?

Cotumaccio¹,

D’Agostino²,

Policriti³

et al. 2021

Preprint

View full text Add to dashboard Cite

Consider the problem of matching a pattern P of length π against the elements of a given regular language L in the setting where L can be pre-processed off-line in a fast data structure (an index). Regular expression matching is an ubiquitous problem in computer science, finding fundamental applications in areas including, but not limited to, natural language processing, search engines, compilers, and databases. Recent results have settled the exact complexity of this problem: Θ(πm) time is necessary and sufficient for indexed pattern matching queries, where m is the size of an NFA recognizing L. This, however, does not mean that all regular languages are hard to index: for instance, for the sub-class of Wheeler languages [SODA'20] we can reduce query time to the optimal O(π). A Wheeler language admits a total order of a finite refinement of its Myhill-Nerode equivalence classes reflecting the co-lexicographic order of their elements. This boosts indexing performance because classes whose elements are suffixed by P form a range in this order. In [SODA'21], this technique was extended to arbitrary NFAs by allowing the order to be partial. This line of attack suggested that the width p of such an order is the parameter ultimately capturing the fine-grained complexity of the problem: (i) indexed pattern matching can always be solved in O(πp 2 ) time, (ii) the standard powerset construction algorithm always produces an output whose size is exponential in p rather than in the input's size, and (iii) p even determines how succinctly NFAs can be encoded.In the present work, we take a step further and study the hierarchy of p-sortable languages: regular languages accepted by automata of width p. Our main contributions are the following: (i) we show that the hierarchy is strict and does not collapse, (ii) we provide (exponential) upper and lower bounds relating the minimum widths of equivalent NFAs and DFAs, and (iii) we characterize DFAs of minimum p for a given L via a co-lexicographic variant of the Myhill-Nerode theorem. Our findings imply that in polynomial time we can build an index breaking the worst-case conditional lower bound of Ω(πm), whenever the input NFA's width is at most ϵ log 2 m, for any constant 0 ≤ ϵ < 1/2.

show abstract

Section: Introductionmentioning

confidence: 78%

Section: :8mentioning

confidence: 97%

Section: :8mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Which Regular Languages can be Efficiently Indexed?

Cotumaccio¹,

D’Agostino²,

Policriti³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Linear Time Construction of Indexable Founder Block Graphs

Mäkinen¹,

Cazaux²,

Equi³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce a compact pangenome representation based on an optimal segmentation concept that aims to reconstruct founder sequences from a multiple sequence alignment (MSA). Such founder sequences have the feature that each row of the MSA is a recombination of the founders. Several linear time dynamic programming algorithms have been previously devised to optimize segmentations that induce founder blocks that then can be concatenated into a set of founder sequences. All possible concatenation orders can be expressed as a founder block graph. We observe a key property of such graphs: if the node labels (founder segments) do not repeat in the paths of the graph, such graphs can be indexed for efficient string matching. We call such graphs segment repeat-free founder block graphs.We give a linear time algorithm to construct a segment repeat-free founder block graph given an MSA. The algorithm combines techniques from the founder segmentation algorithms (Cazaux et al. SPIRE 2019) and fully-functional bidirectional Burrows-Wheeler index (Belazzougui and Cunial, CPM 2019). We derive a succinct index structure to support queries of arbitrary length in the paths of the graph.Experiments on an MSA of SAR-CoV-2 strains are reported. An MSA of size 410 × 29811 is compacted in one minute into a segment repeat-free founder block graph of 3900 nodes and 4440 edges. The maximum length and total length of node labels is 12 and 34968, respectively. The index on the graph takes only 3% of the size of the MSA.

show abstract

On Indexing and Compressing Finite Automata

Cotumaccio¹,

Prezza²

2020

Preprint

View full text Add to dashboard Cite

An index for a finite automaton is a powerful data structure that supports locating paths labeled with a query pattern, thus solving pattern matching on the underlying regular language. The problem is hard in the general case: a recent conditional lower bound suggests that, in the worst case, deciding at query time whether a pattern of length m belongs to the substring closure of the language accepted by A requires Ω(m • |A|) time. On the other hand, Gagie et al. [TCS 2017] introduced a subclass of automata that allow an optimal Õ(m)-time solution based on prefix-sorting the states in a total order. In this paper, we solve the long-standing problem of indexing arbitrary finite automata, matching the above bounds. Our solution consists in finding a partial co-lexicographic order of the states and proving, as in the total order case, that states reached by a given string form one interval on the partial order, thus enabling indexing. We provide a lower bound stating that such an interval requires O(p) words to be represented, p being the order's width (i.e. the size of its largest antichain). Indeed, we show that p determines the complexity of several fundamental problems on finite automata: (i) Letting σ be the alphabet size, we provide an encoding for NFAs using ⌈log σ⌉ + 2⌈log p⌉ + 2 bits per transition and a smaller encoding for DFAs using ⌈log σ⌉ + ⌈log p⌉ + 2 bits per transition. This is achieved by generalizing the Burrows-Wheeler transform to arbitrary automata. (ii) We show that indexed pattern matching can be solved in Õ(m • p 2 ) query time on NFAs. (iii) We provide a polynomial-time algorithm to index DFAs, while matching the optimal value for p.On the other hand, we prove that the problem is NP-hard on NFAs. (iv) We show that, in the worst case, the classic powerset construction algorithm for NFA determinization generates an equivalent DFA of size 2 p (n − p + 1) − 1, where n is the number of NFA's states. Contribution (i) provides a new compression paradigm for labeled graphs. Contributions (ii)-(iii) solve the regular language indexing problem, notably with a polynomial-time solution for DFAs. Contribution (iv) implies a new FPT analysis for the complexity of classic algorithms on automata, including membership and equivalence (the latter being PSPACE-complete when input automata are NFAs).

show abstract

Graphs cannot be indexed in polynomial time for sub-quadratic time string matching, unless SETH fails

Cited by 4 publications

References 34 publications

Which Regular Languages can be Efficiently Indexed?

Which Regular Languages can be Efficiently Indexed?

Linear Time Construction of Indexable Founder Block Graphs

On Indexing and Compressing Finite Automata

Contact Info

Product

Resources

About