In this study, undoped and cobalt‐doped manganese oxide Co:Mn3O4 (0, 2, and 4 at%) are elaborated by the spray pyrolysis procedure on glass substrates. The effect of cobalt doping concentrations on the structural, morphological, and optical features (linear and nonlinear) are investigated through the X‐ray diffractometer, Raman spectroscopy, Scanning Electron Microscopy with an attached energy dispersive X‐ray and UV–Vis‐NIR spectrophotometer. The XRD investigations exhibit a polycrystalline tetragonal structure for all Co:Mn3O4 and confirm by Raman results with the distinctive tetragonal mode A1g at 657 cm−1. The SEM images show an increase in roughness as cobalt doping concentration increases. The EDS quantitative analysis revealed the presence of manganese, oxygen, and cobalt. Furthermore, the optical investigation is pursued firstly through the linear optical that showed a decrease in the transmittance, and optical band gap energy as the cobalt concentration increased due to oxygen vacancies and defects and the increase of refractive index from 2.05 to 2.21. Second, the nonlinear optical parameters show a high value of the nonlinear refractive index 3.75 × 10−7 (esu) and the third‐order susceptibility χ3 = 2.2 × 10−8(esu) of Co:Mn3O4(2at%). These values put the Co:Mn3O4(2at%) thin layer as a candidate in the optoelectronic applications as a third‐order nonlinear medium.