The self-annihilation of dark matter particles with mass in the MeV range can produce gamma rays via prompt or secondary radiation. The annihilation rate for such light dark matter particles is however tightly constrained by cosmic microwave background (CMB) data. Here we explore the possibility of discovering MeV dark matter annihilation with future MeV gamma-ray telescopes taking into account the latest and future CMB constraints. We study the optimal energy window as a function of the dominant annihilation final state. We consider both the (conservative) case of the dwarf spheroidal galaxy Draco and the (more optimistic) case of the Galactic center. We find that for certain channels, including those with one or two monochromatic photon(s) and one or two neutral pion(s), a detectable gamma-ray signal is possible for both targets under consideration, and compatible with CMB constraints. For other annihilation channels, however, including all leptonic annihilation channels and two charged pions, CMB data rule out any significant signal of dark matter annihilation at future MeV gamma-ray telescopes from dwarf galaxies, but possibly not for the Galactic center.PACS numbers: 95.35.+d, 95.85.Pw, 98.52Wz