It is difficult to detect ports in polarimetric SAR images due to the complicated components, morphology, and coastal environment. This paper proposes an unsupervised port detection method by extracting the water of the port based on three-component decomposition and multi-scale thresholding segmentation. Firstly, the polarimetric characteristics of the port water are analyzed using modified three-component decomposition. Secondly, the volume scattering power and the power ratio of the double-bounce scattering power to the volume scattering power (PRDV) are used to extract the port water. Water and land are first separated by a global thresholding segmentation of the volume scattering power, in which the sampling region used for the threshold calculation is automatically selected by a proposed homogeneity measure. The interference water regions in the ports are then separated from the water by segmenting the PRDV using the multi-scale thresholding segmentation method. The regions of interest (ROIs) of the ports are then extracted by determining the connected interference water regions with a large area. Finally, ports are recognized by examining the area ratio of strong scattering pixels to the land in the extracted ROIs. Seven single quad-polarization SAR images acquired by RADARSAT-2 covering the coasts of Dalian, Zhanjiang, Fujian, Tianjin, Lingshui, and Boao in China and Berkeley in America are used to test the proposed method. The experimental results show that all ports are correctly and quickly detected. The false alarm rates are zero, the intersection of union section (IoU) indexes between the detected port and the ground truth can reach 75%, and the average processing time can be less than 100 s.