The collective nature of the variation in host associated microbial communities suggest that they exhibit low dimensional characteristics. To identify these lower dimensional descriptors, we propose SMbiot (pronounced SIM BY OT): a Shared Latent Model for Microbiomes and their hosts. In SMbiot, latent variables embed host-specific microbial communities in a lower dimensional space and the corresponding features reflect controlling axes that dictate community compositions. Using data from different animal hosts, organ sites, and microbial kingdoms of life, we show that SMbiot identifies a small number of host-specific latent variables that accurately capture the compositional variation in host associated microbial communities. By using the same latents to describe hosts' phenotypic states and the host-associated microbiomes, we show that the latent space embedding is informed by host physiology as well as the associated microbiomes. Importantly, SMbiot enables the quantification of host phenotypic differences associated with altered microbial community compositions in a host-specific manner, underscoring the context specificity of host-microbiome associations. SMbiot can also predict missing host metadata or microbial community compositions. This way, SMbiot is a concise quantitative method to understand the low dimensional collective behavior of host-associated microbiomes.