In this work, the terbium oxide (Tb4O7) passivation layers were sputtered using radio frequency (RF) sputtering and then post-annealed in oxygen (O2), nitrogen (N2), argon (Ar), and nitrogen-oxygen-nitrogen (NON) environments. Different characterization techniques were utilized to investigate the detailed influence of these different annealing environments. Grazing incidence X-ray diffraction (GIXRD) patterns indicate a cubic crystal structure in all samples investigated. The sample annealed in Ar ambient reached the highest crystallinity close to 48 nm. The morphological analysis showed a distinct surface structure for all the investigated samples when viewed with field emission scanning electron microscopy (FESEM). Atomic force microscopy (AFM) was utilized to study the surface roughness, and it was found that the Tb4O7 passivation layer annealed in an Ar ambient achieved a higher surface roughness at (1.150 nm). The UV-Vis analysis was performed, and absorbance was determined from reflectance data. The direct bandgap (Eg) was estimated by applying the Kubelka-Munk (KM) approach and found to be 3.28, 3.17, 2.37, and 2.27 eV for O2, N2, Ar, and NON ambients, respectively. These findings highlight the importance of post-deposition annealing treatments using different annealing gases and provide insight into the development of optimal Tb4O7 passivation layers.