The rapidly developing spatial omics techniques generate datasets with diverse scales and modalities. However, most existing methods focus on modeling dynamics of single cells while ignore microenvironments (MEs) which bridge single cells to tissues. Here we present SOTIP (Spatial Omics mulTIPle-task analysis), a scalable framework incorporating MEs and their interrelationships into a unified graph. Based on this graph, three tasks can be performed, including spatial heterogeneity (SHN) quantification, spatial domain (SDM) identification and differential microenvironment (DME) analysis. We validate SOTIP’s accuracy, robustness, interpretation, and scalability on various datasets by comparing with state-of-art methods. In mammalian cerebral cortex, we reveal a striking gradient SHN pattern with strong correlations with the cortical depth. In human triple-negative breast cancer (TNBC), we identify previously unreported molecular polarizations around SOTIP-detected tumor-immune boundaries. Most importantly, we discover TNBC subtype-specific MEs, which exhibit interesting compositional and spatial properties, and could be powerful clinical indicators. Overall, by modeling biologically explainable microenvironments, SOTIP outperforms state-of-art methods and provides new perspectives for data interpretation, which facilitates further understanding of spatial information on a variety of biological issues.