The study focused on the application of speckle tracking algorithm in the segmentation of cardiac color ultrasound images of patients with atrial fibrillation combined with heart failure. First, the optical flow method and block matching method were introduced on the basis of multiphase level set algorithm. Then, the pyramid block matching method was applied to build a pyramid model from bottom to top according to each image, and thus a new segmentation algorithm of cardiac color ultrasound image was constructed. The speckle tracking algorithm based on the pyramid block matching method was applied to segment cardiac color ultrasound images of 136 patients with atrial fibrillation and heart failure and compared with the traditional diagnosis for the sensitivity, specificity, and accuracy. It was found that the curve smoothness and accuracy of the algorithm in this study were better than the traditional level set algorithm, and it made up for the shortcomings of the traditional method. The proportion of patients of class III-IV cardiac function was significantly higher than that of non-atrial fibrillation patients, and the difference was statistically significant (
P
<
0.05
); patients of classes III-IV showed better left ventricular ejection fraction (LVEF) (42.4 ± 2.8%), left ventricular end-diastolic diameter (LVED) (58.7 ± 7.4 mm), left ventricular end-systolic diameter (LVSD) (49.3 ± 5.6 mm), and left atrial inner diameter (LAD) (55.0 ± 1.4 mm) versus those of classes I-II, of whom the corresponding indexes were 58.8 ± 3.3%, 48.5 ± 5.9 mm, 33.5 ± 4.5 mm, and 45.2 ± 2.0 mm. The accuracy of diagnosis based on the algorithm of this study (93.22%) was significantly higher than that of traditional method (79.23%), and the differences were statistically significant (
P
<
0.05
). In conclusion, the algorithm in this study improves the segmentation accuracy and smoothness of the curve, which is suggested in clinic.