Existing visible light positioning (VLP) algorithms typically require multiple LEDs to achieve high-precision positioning. However, in practice, the number of available LEDs is influenced by various factors such as the layout of LEDs and the field-of-view of the receiver, and these factors can significantly affect the coverage of a VLP algorithm. To address this issue, this paper proposes an inertial sensor-assisted single view geometry (ISA-SVG) algorithm, which can achieve accurate positioning as little as one LED. In particular, we first establish the projection transformation model between the actual point and its projection on the image plane. Then, a pose fusion model that effectively fuses the inertial sensor information and VLP information is established. Finally, the position of the receiver based on the pose fusion model is estimated. IAS-SVG significantly improves the usability and coverage performance of visible light positioning algorithms. Simulation results show that the coverage rate of the ISA-SVG algorithm is over 95%. Experimental results show that the proposed method can achieve an average positioning error as low as 4.70 cm.