On the Separation Question for Tree Languages

Abstract

We show that the separation property fails for the classes Σ n of the RabinMostowski index hierarchy of alternating automata on infinite trees. This extends our previous result (obtained with Szczepan Hummel) on the failure of the separation property for the class Σ 2 (i.e., for co-Büchi sets). The non-separation result is also adapted to the analogous classes induced by weak alternating automata.To prove our main result, we first consider the Rabin-Mostowski index hierarchy of deterministic automata on infinite words, for which we give a complete answer (generalizing previous results of Selivanov): the separation property holds for Π n and fails for Σ n -classes. The construction invented for words turns out to be useful for trees via a suitable game.It remains open if the separation property holds for all classes Π n of the index hierarchy for tree automata. To give a positive answer it would be enough to show the reduction property of the dual classes-a method well-known in descriptive set theory. We show that it cannot work here, because the reduction property fails for all classes in the index hierarchy.

Separation Property for wB- and wS-regular Languages

“…The above theorem is extended for higher levels of the alternating index hierarchy in [AMN12]. Theorem 1.4 relies on the fact that every alternating (1, 2)-parity tree automaton is equivalent to a nondeterministic (1, 2)-parity tree automaton.…”