Evaluating tissue architecture from routine hematoxylin and eosin-stained (H&E) slides is prone to subjectivity and sampling bias. Here, we extensively annotated ~40,000 images of five tissue texture types and ~25,000 images of lymphocyte quantity to train deep learning models. We defined histopathological patterns in over 400 clear-cell renal cell carcinoma H&E-stained slides of The Cancer Genome Atlas (TCGA) and resolved sampling and staining differences by harmonizing textural composition. By integrating multi-omic and imaging data, we profiled their clinical, immunological, genomic, and transcriptomic phenotypes. Histological grade, stage, adaptive immunity, the epithelial-to-mesenchymal transition signature and lower mutation burden were more common in stroma-rich samples. Histological proximity between the malignant and normal renal tissues was associated with poor survival, cellular proliferation, tumor heterogeneity, and wild-type PBRM1. This study highlights textural characterization to standardize sampling differences, quantify lymphocyte infiltration and discover novel histopathological associations both in the intratumoral and peritumoral regions.