Total focus (TFM) ultrasonic inspection imaging only uses the amplitude of defect data for delayed stack imaging (DAS), ignoring the phase information in the echo signal, which easily leads to low imaging resolution and interferes with subsequent quantitative analysis of defect size. To solve these problems, this paper proposes a phase coherence factor (TF-PCF) weighted full-focus imaging method based on threshold fusion: by combining the image sound intensity matrix with mathematical statistics, a reasonable threshold is set to screen out the effective pixels, and the parameter selection rules of the PCI algorithm model are redefined. The simulation and experimental results show that the TF-PCF imaging algorithm can effectively eliminate the noise information in the imaging area, including the artifacts near the defects that are difficult to be processed by PCI. Compared with PCI, TF-PCF has higher ability to quantify defect size. When using p-wave to detect large objects with a large number of defects and complex location distribution, the lateral size quantization error of the TF-PCF imaging algorithm is reduced by 8.5%. When sampling shear wave imaging to detect defects with tilted wedges, the lateral and longitudinal size quantization errors of defects are reduced by 10% and 20%, respectively. The reliability of defect size quantitative analysis is improved. Under the same test conditions, the complexity of TF-PCF model is far less than that of PCI model, which greatly improves the practicability of the algorithm model.