One of the main goals in the study of color symmetry is to classify colorings of symmetrical objects through their color groups. The term color group is taken to mean the subgroup of the symmetry group of the uncolored symmetrical object which induces a permutation of colors in the coloring. This work looks for methods of determining the color group of a colored symmetric object. It begins with an index n subgroup H of the symmetry group G of the uncolored object. It then considers H-invariant colorings of the object, so that the color group H(*) will be a subgroup of G containing H. In other words, H ≤ H(*) ≤ G. It proceeds to give necessary and sufficient conditions for the equality of H(*) and G. If H(*) ≠ G and n is prime, then H(*) = H. On the other hand, if H(*) ≠ G and n is not prime, methods are discussed to determine whether H(*) is G, H or some intermediate subgroup between H and G.