Laser powder bed fusion (LPBF) is an additive manufacturing technology with high practical value. In order to improve the quality of the fabricated parts, process monitoring has become a crucial solution, offering the potential to ensure manufacturing stability and repeatability. However, a cardinal challenge involves discerning a precise correlation between process characteristics and potential defects. This paper elucidates the integration of an off-axis vision monitoring mechanism via a high-speed camera focused on capturing the single-track melting phenomenon. An innovative image processing method was devised to segment the plume and spatters, while Kalman filter was employed for multi-object tracking of the spatters. The features of both the plume and spatters were extracted, and their relationship with molten states was investigated. Finally, the PSO-XGBoost algorithm was utilized to identify five molten states, achieving an accuracy of 92.16%. The novelty of this approach resides in its unique combination of plume characteristics, spatter features, and computationally efficient machine learning models, which collectively address the challenge of limited field of view prevalent in real production scenarios, thereby enhancing process monitoring efficacy. Relative to existing methodologies, the proposed PSO-XGBoost approach offers heightened accuracy, convenience, and appropriateness for the monitoring of the LPBF process. This work provides an effective and novel approach to monitor the LPBF process and evaluate the part fabrication quality for complex and changeable working conditions.