Single crystals of Tb3+ single-doped and Tb3+/Pr3+ co-doped CaYAlO4 were produced by the Czochralski method. The room-temperature polarized absorption spectra, emission spectra, and decay curves were recorded and analyzed in detail. The absorption cross-section around 487 nm was found to be 1.53 × 10−22 cm2 for the π polarization in the Tb3+:CaYAlO4 crystal and increased to 5.23 × 10−22 cm2 in the Tb3+/Pr3+:CaYAlO4 crystal. The spectroscopic parameters were calculated through the Judd–Ofelt theory. For the Tb3+:CaYAlO4 crystal, the emission bands of green light at 546 nm and yellow light at 587 nm had fluorescence branching ratios of 64.7% and 6.65% with cross-sections of 8.82 × 10−22 cm2 (σ-polarization) and 0.44 × 10−22 cm2 (π-polarization), respectively. The decay lifetimes of 5D4 multiplets were measured to be 1.41 ms and 1.1 ms for Tb3+:CaYAlO4 and Tb3+/Pr3+:CaYAlO4 crystals, respectively. The energy transfer mechanisms of Tb3+ and Pr3+ and their emission spectral intensities at different temperatures were analyzed. As the temperature increased, the luminescence intensity of the Tb3+:CaYAlO4 and Tb3+/Pr3+:CaYAlO4 crystals decreased almost linearly with the CIE coordinate variation, from (0.370, 0.621) to (0.343, 0.636) and from (0.345, 0.638) to (0.246, 0.698), respectively. The results indicate the potential of Tb3+:CaYAlO4 and Tb3+/Pr3+:CaYAlO4 crystals as visible laser materials with a wide temperature range.