In this study, Yb:YAG-nanocrystal-doped silica glass with high transmission and excellent spectral properties was successfully prepared using a modified sol–gel method. The X-ray diffraction (XRD), micro-Raman spectroscopy, electron paramagnetic resonance (EPR), transmission electron microscopy (TEM), and high-resolution TEM (HR-TEM) analyses confirmed that the Yb:YAG nanocrystals, with their low content, homogeneous distribution, and small crystal size, directly crystallized into the silica glass network without annealing treatment. In contrast with conventional microcrystalline glass having large particles (>0.1 μm) and a large particle content, nanocrystalline glass with a homogeneous distribution and sizes of ~22 nm had higher optical transmittance and better spectral properties. Compared with Yb3+ doped silica glass without nanocrystals, the Yb:YAG-nanocrystal-doped silica glass had a 28% increase in absorption cross-section at 975 nm and a 172% enhanced emission cross-section at 1030 nm without any changes in the spectral pattern of the Yb3+ ions in the silica glass. Meanwhile, the Yb:YAG-doped silica glass with large size and high optical quality was easily prepared. Therefore, the Yb:YAG-nanocrystal-doped silica glass is expected to be a promising near-infrared laser material.