Phenotypic variation can lead to variation in the strength and outcome of species interactions. Variation in phenotypic traits can arise due to plastic responses to environmental stimuli, underlying genetic variation, or both, and may reflect differences in the focal organism or aspects of the extended phenotype (e.g., associated microbes). We used a reciprocal transplant experiment of Porites corals to evaluate the role of plasticity vs. heritable diversity on phenotypic traits and performance of corals that varied in their prior exposure to vermetid gastropods, an organism known to reduce coral growth and survival. We measured a suite of phenotypic traits associated with coral performance, many of which showed a plastic response to vermetid exposure. Vermetids decreased calcification of corals, increased microbial diversity, and shifted microbial composition. Most traits also showed a signature of previous exposure environment that persisted even when exposure was reversed: i.e., under the same conditions, corals naïve to vermetids had slower calcification rates, thicker tissues, higher Symbiodiniaceae densities, and different microbiomes than corals previously exposed to vermetids. We suggest the phenotypic differences are heritable, as reefs with and without vermetids were comprised of two different mitotypes, that revealed high, consistent genetic variation. Vermetids were only found on the fast‐growing coral mitotype that was characterized by thin tissue, and that likely had a history of disturbance. As extended phenotypes can have community impacts, we suggest vermetid, in addition to microbes, are part of the extended community phenotype of these corals. Coral genotypes can establish different reef trajectories, with thin‐tissue types more prone to disturbance and subsequent colonization by other species, like vermetids, which can further facilitate the degradation of coral reefs. The effects of the extended phenotype of species likely influence heterogeneity across landscapes as well as temporal differences in community composition.