For a finitely generated group, there are two recent generalizations of the notion of a quasiconvex subgroup of a word-hyperbolic group, namely a stable subgroup and a Morse or strongly quasiconvex subgroup. Durham and Taylor [DT15] defined stability and proved stability is equivalent to convex cocompactness in mapping class groups. Another natural generalization of quasiconvexity is given by the notion of a Morse or strongly quasiconvex subgroup of a finitely generated group, studied recently by Tran [T17] and Genevois [G17]. In general, a subgroup is stable if and only if the subgroup is Morse and hyperbolic. In this paper, we prove that two properties of being Morse and stable coincide for a subgroup of infinite index in the mapping class group of an oriented, connected, finite type surface with negative Euler characteristic.

The notions of stable and Morse subgroups of finitely generated groups generalize the concept of a quasiconvex subgroup of a word-hyperbolic group. For a word-hyperbolic group G, Kapovich [K96] provided a partial algorithm which, on input a finite set S of G, halts if S generates a quasiconvex subgroup of G and runs forever otherwise. In this paper, we give various detection and decidability algorithms for stability and Morseness of a finitely generated subgroup of mapping class groups, right-angled Artin groups, toral relatively hyperbolic groups, and finitely generated groups discriminated by a locally quasiconvex torsion-free hyperbolic group (for example, ordinary limit groups).