Biodiversity patterns are shaped by the combination of dispersal, environment, and stochasticity, but how the influence of these drivers changes in fragmented habitats remains poorly understood. We examined patterns and relationships among total (γ) and site‐level (α) diversity, and site‐to‐site variation in composition (β‐diversity) of tree communities in structurally contiguous and fragmented tropical rainforests within a human‐modified landscape in India's Western Ghats. First, for the entire landscape, we assessed the extent to which habitat type (fragment or contiguous forest), space and environment explained variation in α‐diversity and composition. Next, within fragments and contiguous forest, we assessed the relative contribution of spatial proximity, environmental similarity, and their joint effects in explaining β‐diversity. We repeated these assessments with β‐diversity values corrected for the confounding effects of α‐ and γ‐diversity using null models (β‐deviation). Lower γ‐diversity of fragments resulted from both lower α‐ and β‐diversity compared to contiguous forests. However, β‐deviation did not differ between contiguous forests and fragments. Fragmented and contiguous forest clearly diverged in floristic composition, which was attributable to β‐diversity being driven by differences in elevation and MAP. Within fragmented forest, neither space nor environment explained β‐diversity, but β‐deviation increased with greater elevational differences. In contiguous forests by comparison, environment alone (mainly elevation) explained the most variation in β‐diversity and β‐deviation of both species' occurrences and abundances. Spatial gradients in environmental conditions played a larger role than dispersal limitation in shaping diversity and composition of tree communities across forest fragments. Thus, location of remnant patches at different elevations was a key factor underlying site‐to‐site variability in species abundances across fragments. Understanding the environmental characteristics of remnant forests in human‐modified landscapes, combined with knowledge of species–environment relationships across different functional groups, would therefore be important considerations for management and restoration planning in human‐modified landscapes.