Near-infrared photodetectors with polarization-sensitive capabilities have garnered significant attention in modern optoelectronic devices. SbSeI, one of the quasi-1D ternary V–VI–VII compounds, exhibits enormous advantages in near-infrared polarization detection due to its suitable bandgap and pronounced anisotropy. Here, SbSeI nanowires were obtained by a mechanical exfoliation method from the bulk crystals, and the photoelectric properties and anisotropy were systematically investigated. The as-fabricated photodetector exhibits a wide spectral photoresponse range from visible to near-infrared (445–980 nm), recording a responsivity of 825.0 mA/W and a specific detectivity of 6.9 × 1010 Jones. Importantly, the strong anisotropy of phonon vibrations was demonstrated via angle-resolved polarization Raman spectroscopy, and the photodetector exhibits a photocurrent dichroic ratio up to 1.69 at 980 nm. These results reveal that SbSeI is a highly in-plane anisotropic semiconductor and a promising candidate material for high-performance broadband polarization-sensitive photodetectors. Our work also sheds light on the future research interest in the group V–VI–VII semiconductors.