The outer automorphism group Out(G) of a group G acts on the set of conjugacy classes of elements of G. McCool proved that the stabilizer Mc(C) of a finite set of conjugacy classes is finitely presented when G is free. More generally, we consider the group Mc(H) of outer automorphisms Φ of G acting trivially on a family of subgroups H i , in the sense that Φ has representatives α i with α i equal to the identity on H i .When G is a toral relatively hyperbolic group, we show that these two definitions lead to the same subgroups of Out(G), which we call "McCool groups" of G. We prove that such McCool groups are of type VF (some finite index subgroup has a finite classifying space). Being of type VF also holds for the group of automorphisms of G preserving a splitting of G over abelian groups.We show that McCool groups satisfy a uniform chain condition: there is a bound, depending only on G, for the length of a strictly decreasing sequence of McCool groups of G. Similarly, fixed subgroups of automorphisms of G satisfy a uniform chain condition.