There is observational evidence for inside-out growth of giant elliptical galaxies since z 2 − 3, which is -in contrast to disk galaxies -not driven by in-situ star formation. Many of the ∼ 10 11 M ⊙ systems at high redshift have small sizes ∼ 1kpc and surface brightness profiles with low Sersic indices n. The most likely descendants at z = 0 have, on average, grown by a factor of two in mass and a factor of four in size, indicating r ∝ M α with α 2. They also have surface brightness profiles with n 5. This evolution can be qualitatively explained on the basis of two assumptions: compact ellipticals predominantly grow by collisionless minor (mass-ratio 1:10) or intermediate (mass-ratio 1:5) 'dry' mergers, and they are embedded in massive dark matter halos which support the stripping of merging satellite stars at large radii. We draw these conclusions from idealized collisionless mergers spheroidal galaxies -with and without dark matter -with mass ratios of 1:1, 1:5, and 1:10. The sizes evolve as r ∝ M α with α < 2 for mass-ratios of 1:1 (and 1:5 without dark matter halos) and , while doubling the stellar mass, the Sersic index increases from n ∼ 4 to n ∼ 5. For minor mergers of galaxies embedded in dark matter halos, the sizes grow significantly faster and the profile shapes change more rapidly. Surprisingly, already mergers with moderate massratios of 1:5, well motivated by recent cosmological simulations, give α ∼ 2.3 and after only two merger generations (∼ 40 per cent added stellar mass) the Sersic index has increased to n > 8 (n ∼ 5.5 without dark matter), reaching a final value of n = 9.5 after doubling the stellar mass. This is accompanied by a significant increase of the dark matter fraction (from ∼ 40 per cent to 70 per cent) within the stellar halfmass radius, driven by the strong size increase probing larger, dark matter dominated regions. For equal-mass mergers the effect is much weaker. We conclude that only a few intermediate mass-ratio mergers (∼ 3 − 5 with initial mass-ratios of 1:5) of galaxies embedded in massive dark matter halos can result in the observed concurrent inside-out growth and the rapid evolution in profile shapes. This process might explain the existence of present day giant ellipticals with sizes, r > 4kpc, high Sersic indices, n > 5, and a significant amount of dark matter within the half-light radius. Apart from negative stellar metallicity gradients such a 'minor' merger scenario also predicts significantly lower dark matter fractions for z ∼ 2 compact quiescent galaxies and their rare present day analogues.