Ternary chalcogenide PbGa2Se4 with high resistivity, photosensitivity, and excellent nonlinear properties, exhibits a potential application in optoelectronic and nonlinear optical devices. However, the preparation of large‐sized pure PbGa2Se4 crystals is challenging due to the presence of peritectic reaction and a narrow homogeneity region. Here, a “quenching‐annealing” method (quenching at 850 °C in ice water, and then annealing at 650 °C for 250 h by reheating) is developed to eliminate the PbSe second phase during the synthesis. Subsequently, the PbGa2Se4 single crystals are successfully grown using chemical vapor transport (CVT) with the I2 as the transport agent. The resulting crystal exhibits the crystal structure belonging to Fddd space group with the lattice parameters of a = 12.7192 Å, b = 21.2831 Å, and c = 21.5226 Å. Additionally, it possesses a wide bandgap (≈2.26 eV), high resistivity (6.59 × 1012 Ω·cm), and defect density calculated via space charge limited current measurement (SCLC) as 2.46 × 1011 cm−3. Photodetectors based on these as‐grown crystals demonstrate exceptional photosensitivity along with a high detectivity (3.2 × 108 Jones).