High-power fiber laser welding has a broad range of applications in industrial processing and modern intelligent manufacturing, but how to assess the quality of laser welding has always been a concern. In the welding process, the geometric feature of the keyhole generated by the high temperature of laser can directly reflect quality of the laser welding. By using machine vision to acquire images in real time and analyze related features, the quality of laser welding can be evaluated, which reduce the cost and time in later inspection. Nevertheless, due to the dynamic complexity of the keyhole as well as the blurring of the keyhole and full penetration hole boundary caused by metal vapor and spatter, identification of multiple type defects through machine vision is still a problem that requires a solution. This paper presents a novel laser welding defect classification method using keyhole boundary-based feature based on machine vision and Hidden Markov process. After effective segmenting the collected welding image, the shapes of the keyhole as well as the full penetration hole were automatically extracted using the characteristics of gray projection distribution and the Poisson extinction method. Subsequently, a pre-trained Hidden Markov Model was employed to establish the connection between the keyhole's geometry and the welding quality defects. Experiments indicate that our theory can efficiently and accurately extract the key geometric shapes. Welding experimental data involving stainless steel 304 verifies the feasibility of the classification theory, which can monitor welding quality and reveal potential porosity and penetration defects.