Electric field control of magnetization and anisotropy in layered structures with perpendicular magnetic anisotropy is expected to increase the versatility of spintronic devices. As a model system for reversible voltage induced changes of magnetism by magnetoionic effects, we present several oxide/metal heterostructures polarized in an electrolyte. Room temperature magnetization of Fe-O/Fe layers can be changed by 64% when applying only a few volts in 1M KOH. In a next step, the bottom interface of the in-plane magnetized Fe layer is functionalized by an L10 FePt(001) underlayer exhibiting perpendicular magnetic anisotropy. During subsequent electrocrystallization and electrooxidation, well defined epitaxial Fe3O4/Fe/FePt heterostructures evolve. The application of different voltages leads to a thickness change of the Fe layer sandwiched between Fe-O and FePt. At the point of transition between rigid magnet and exchange spring magnet regime for the Fe/FePt bilayer, this induces a large variation of magnetic anisotropy.