The spectrum confocal displacement sensor is an innovative type of photoelectric sensor. The non-contact advantages of this method include the capacity to obtain highly accurate measurements without inflicting any harm as well as the ability to determine the object’s surface contour recovery by reconstructing the measurement data. Consequently, it has been widely used in the field of three-dimensional topographic measuring. The spectral confocal displacement sensor consists of a light source, a dispersive objective, and an imaging spectrometer. The scanning mode can be categorized into point scanning and line scanning. Point scanning is inherently present when the scanning efficiency is low, resulting in a slower measurement speed. Further improvements are necessary in the research on the line-scanning type. It is crucial to expand the measurement range of existing studies to overcome the limitations encountered during the detection process. The objective of this study is to overcome the constraints of the existing line-swept spectral confocal displacement sensor’s limited measuring range and lack of theoretical foundation for the entire system. This is accomplished by suggesting an appropriate approach for creating the optical design of the dispersive objective lens in the line-swept spectral confocal displacement sensor. Additionally, prism-grating beam splitting is employed to simulate and analyze the imaging spectrometer’s back end. The combination of a prism and a grating eliminates the spectral line bending that occurs in the imaging spectrometer. The results indicate that a complete optical pathway for the line-scanning spectral confocal displacement sensor has been built, achieving an axial resolution of 0.8 μm, a scanning line length of 24 mm, and a dispersion range of 3.9 mm. This sensor significantly expands the range of measurements and fills a previously unaddressed gap in the field of analyzing the current stage of line-scanning spectral confocal displacement sensors. This is a groundbreaking achievement for both the sensor itself and the field it operates in. The line-scanning spectral confocal displacement sensor’s design addresses a previously unmet need in systematic analysis by successfully obtaining a wide measuring range. This provides systematic theoretical backing for the advancement of the sensor, which has potential applications in the industrial detection of various ranges and complicated objects.