To simultaneously acquire the spectral and polarization information of the target and achieve the monitoring and identification of the target object, a polarization spectral integrated imaging system is proposed in this paper. Firstly, the structural principle of the polarization spectral integrated imaging system is introduced. The relationship between the spatial resolution, spectral resolution, and the system’s structural parameters is analyzed. The design of the optical part of the polarization spectral integrated imaging system is completed, along with the tolerance analysis. Secondly, the mechanical structure of the polarization spectral integrated imaging system is designed. Finally, by using a drone to carry the polarization spectral integrated imaging system, a simulation experiment for sea surface oil spill monitoring is conducted, and the hyperspectral and polarization information of the ocean, crude oil, fuel oil, palm oil, diesel, and gasoline are obtained. The polarization and spectral information were integrated. The integration of hyperspectral and polarization data yields remarkable enhancement outcomes, allowing for the clear delineation of previously challenging-to-identify crude oil contamination areas against the marine background in the fused images, characterized by sharper boundaries and improved discriminability. This accomplishment underscores the feasibility of our system for the rapid identification of large-scale oil spill events.