An improved particle filter algorithm based on Cam-Shift algorithm applied to outdoor visible light communication (outdoor-VLC) is presented in this paper. In the outdoor-VLC system, accurately and completely tracking and extracting the target signal source Light Emitting Diode (LED) area is the premise for realizing communication. However, few existing studies pay attention to it. In the dynamic outdoor environment, there will be a lot of different environmental interferences (such as background interference, similar object interference, etc.), which greatly increases the difficulty of tracking and extracting the target signal source LED area. Therefore, in this paper, an improved tracking algorithm is proposed to solve the problem of how to track and extract the target signal source LED area accurately, stably and in real time in the outdoor-VLC system under various environmental interferences, so as to increase the feasibility of practical application of VLC system in outdoor scenes. This improved algorithm combines the particle filter algorithm and Cam-Shift algorithm originally. Experimental results show that the proposed algorithm has good accuracy, robustness and real-time performance under the environment of multiple interference factors. Accordingly, the proposed algorithm can be applied to the outdoor-VLC system with various environmental interferences, and can realize the actual first step of communication in VLC system based on image sensor well, laying a foundation for feature extraction, data transmission and other subsequent steps.