Rhodoliths built by crustose coralline algae (CCA) are ecosystem engineers of global importance. In the Arctic photic zone, their three‐dimensional growth emulates the habitat complexity of coral reefs but with a far slower growth rate, growing at micrometers per year rather than millimeters. While climate change is known to exert various impacts on the CCA's calcite skeleton, including geochemical and structural alterations, field observations of net growth over decade‐long timescales are lacking. Here, we use a temporally explicit model to show that rising ocean temperatures over nearly 100 years were associated with reduced rhodolith growth at different depths in the Arctic. Over the past 90 years, the median growth rate was 85 μm year−1 but each °C increase in summer seawater temperature decreased growth by a mean of 8.9 μm (95% confidence intervals = 1.32–16.60 μm °C−1, p < .05). The decrease was expressed for rhodolith occurrences in 11 and 27 m water depth but not at 46 m, also having the shortest time series (1991–2015). Although increasing temperatures can spur plant growth, we suggest anthropogenic climate change has either exceeded the population thermal optimum for these CCA, or synergistic effects of warming, ocean acidification, and/or increasing turbidity impair rhodolith growth. Rhodoliths built by calcitic CCA are important habitat providers worldwide, so decreased growth would lead to yet another facet of anthropogenic habitat loss.