A novel magnetic spectrometer for Inverse-Compton X-ray source is proposed. Compton recoil electrons are generated by a lithium converter, and then confined by a complex collimator and spectrally resolved by a sector-shaped double-focusing magnet. A method of optimization for the converter is investigated, and the dependence of the best energy resolution on converting efficiency is quantitatively revealed. The configuration of the magnet is specially designed to cover a wide range of electron energy and to achieve a large collecting solid angle. The efficiency and relative energy resolution of the designed spectrometer, according to Monte-Carlo simulation using Geant4, are 10-4 e/p and about 5% respectively for 3 MeV photons.