We develop a theoretical method within the framework of relativistic many-body theory to accurately treat correlation corrections in atoms with few valence electrons. This method combines the all-order approach currently used in precision calculations of properties of monovalent atoms with the configuration-interaction approach that is applicable for many-electron systems. The method is applied to Mg, Ca, Sr, Zn, Cd, Ba, and Hg to evaluate ionization energies and low-lying energy levels.