Samarium oxide (SMO), a rare-earth oxide, has gathered great interest from researchers because of its variable valences and promising ionic conductivity. Herein, SMOs with cubic and monoclinic phases were synthesized. The changes in the crystal structure of SMOs with sintering temperature were analyzed. The cell based on cubic phase SMO achieves an excellent maximum power density of 0.876 W cm−2 along with a high ionic conductivity at 550 °C, indicating an enhanced ionic conductivity compared with monoclinic phase SMO. Further analysis of x-ray diffraction and x-ray photoelectron spectra results confirmed that there were more oxygen vacancies formed in cubic phase SMO than monoclinic phase SMO, thereby offering more active sites for fast ion transport. Furthermore, both cubic phase and monoclinic phase SMOs are dominated by proton conduction, while cubic phase SMO is further coupled with oxygen ion conduction, which leads to higher ionic conductivity of cubic phase SMO. In this study, the ionic conductivities of SMOs with different crystal structures are compared and reasons for the differences are disclosed, which provides guidance for further applications of SMO.