Deep learning has achieved remarkable progress in medical image analysis, but its effectiveness heavily relies on large-scale and well-annotated datasets. However, assembling a large-scale dataset of annotated histopathological images is challenging due to their unique characteristics, including various image sizes, multiple cancer types, and staining variations. Moreover, strict data privacy in medicine severely restricts data sharing and poses significant challenges in acquiring large-scale and well-annotated histopathological images. To tackle these constraints, Transfer Learning (TL) provides a promising solution by exploiting knowledge from another domain. This study proposes the Uncertainty-guided asymmetric Consistency Domain Adaptation (UCDA), which does not require accessing the source data and is composed of two essential components, e.g., Uncertainty-guided Source-free Transfer Learning (USTL) and Asymmetric Consistency Learning (ACL). In detail, USTL facilitates a secure mapping of the source domain model’s feature space onto the target domain, eliminating the dependency on source domain data to protect data privacy. At the same time, the ACL module measures the symmetry and asymmetry between the source and target domains, bridging the information gap and preserving inter-domain differences among medical images. We comprehensively evaluate the effectiveness of UCDA on three widely recognized and publicly available datasets, namely NCTCRC-HE-100K, PCam, and LC25000. Impressively, our proposed method achieves remarkable performance on accuracy and F1-scores. Additionally, feature visualizations effectively demonstrate the exceptional generalizability and discriminative power of the learned representations. These compelling results underscore the significant potential of UCDA in driving the advancement of deep learning techniques within the realm of histopathological image analysis.