We conducted research into the visible light detection of conical space targets and proposed a method to determine the caliber of a space-based detection system. We constructed a three-dimensional conical space target radiation transfer model and analyzed the effect of the phase angle between the solar incident direction, the detector’s bore sight direction, and the conical target inclination on the radiation intensity of the space target received by the detector. Finally, we obtained the threshold caliber which can meet the detection requirements of the system with the system signal-to-noise ratio as the evaluation standard, and we took the SBV sensor as an example to verify that the caliber determination method in this paper is effective.