In this study, we present a novel material platform based on tantalum-niobium pentoxide (TaNbO5) for integrated photonics applications. TaNbO5 demonstrates exceptional attributes suitable for both linear and nonlinear optics across a wide range of near-infrared wavelengths. Our analysis of the TaNbO5 unit cell revealed crucial lattice parameters and a direct band gap value of 2.268 eV. At a wavelength of 1550 nm, TaNbO5 exhibits a refractive index of 2.22, an extinction coefficient of 5.24×10-4, and other optical properties. We determine a 0.8 μm cut-off core width for single-mode TaNbO5 waveguides, which achieve efficiencies exceeding 99% for single-mode configurations and 98% for multimode structures. When coupled with conventional waveguides, TaNbO5 waveguides demonstrate excellent transmission characteristics. Notably, at a wavelength of 1.55 μm, the single-mode waveguide exhibits minimal excess loss. These findings highlight the significant potential of TaNbO5 waveguides for various near-infrared applications, emphasizing their versatility and promising performance.