The rupture of unstable plaques is a major cause of acute cardiovascular events. The early assessment of carotid plaques can significantly reduce the cardiovascular risks, so developing evaluation models suitable for data from different centers is of great clinical importance. This study leverages plaque datasets from multiple centers to develop a Weighted multi-source carotid artery plaque Unsupervised Classification Framework (WUCF). The multi-source domain adaptation module of the WUCF focuses on maintaining feature consistency between each independent source and target center, while also integrating a specialized domain discriminator expert. This ensures that the knowledge from each source center is effectively learned and combined for accurate predictions in the target domain. The experimental evaluation of WUCF, using datasets from three centers, has demonstrated the method’s superiority and robustness.