Abstract. We study the size of the isometry group Isom(M, g) of Riemannian manifolds (M, g) as g varies. For M not admitting a circle action, we show that the order of Isom(M, g) can be universally bounded in terms of the bounds on Ricci curvature, diameter, and injectivity radius of M . This generalizes results known for negative Ricci curvature to all manifolds.More generally we establish a similar universal bound on the index of the deck group π1(M ) in the isometry group Isom( M , g) of the universal cover M in the absence of suitable actions by connected groups. We apply this to characterize locally symmetric spaces by their symmetry in covers. This proves a conjecture of Farb and Weinberger with the additional assumption of bounds on curvature, diameter, and injectivity radius. Further we generalize results of Kazhdan-Margulis and Gromov on minimal orbifolds of nonpositively curved manifolds to arbitrary manifolds with only a purely topological assumption.