Ferrimagnetic alloy thin films that exhibit perpendicular (out-of-plane) magnetic anisotropy (PMA) with low saturation magnetization, such as GdCo and Mn4N, were predicted to be favorable for hosting small Néel skyrmions for room temperature applications. Due to the exponential decay of interfacial Dzyaloshinskii-Moriya interaction (DMI) and the limited range of spin-orbit-torques, which can be used to drive skyrmion motion, the thickness of the ferrimagnetic layer has to be small, preferably under 20 nm. While there are examples of sub-20 nm, rare earth-transition metal (RE-TM), ferrimagnetic thin films fabricated by sputter deposition, to date rare-earth-free sub-20 nm Mn4N films with PMA have only been reported to be achieved by molecular beam epitaxy, which is not suitable for massive production. Here we report the successful thermal growth of sub-20 nm Mn4N films with PMA at 400-450 °C substrate temperatures on MgO substrates by reactive sputtering. The Mn4N films were achieved by reducing the surface roughness of MgO substrate through a high-temperature vacuum annealing process. The optimal films showed low saturation magnetization (Ms = 43 emu/cc), low magnetic anisotropy energy (0.7 Merg/cc), and a remanent magnetization to saturation magnetization ratio (Mr/Ms) near 1 at room temperature. Preliminary ab-initio density functional theory (DFT) calculations have confirmed the ferrimagnetic ground state of Mn4N grown on MgO. The magnetic properties, along with the high thermal stability of Mn4N thin films in comparison with RE-TM thin films, provide the platform for future studies of practical skyrmion-based spintronic materials.