Controlling the thermal expansion of ceramic materials is important for many of their applications that involve high-temperature processing and/or working conditions. In this study, we investigate the thermal expansion properties of additively manufactured alumina that is reinforced with boron nitride nanotubes (BNNTs) over a broad temperature range, from room temperature to 900 °C. The coefficient of thermal expansion (CTE) of the BNNT-alumina nanocomposite increases with temperature but decreases with an increase in BNNT loading. The introduction of 0.6% BNNTs results in an approximate 16% reduction in the CTE of alumina. The observed significant CTE reduction of ceramics is attributed to the BNNT’s low CTE and ultrahigh Young’s modulus, and effective interfacial load transfer at the BNNT-ceramic interface. Micromechanical analysis, based on in situ Raman measurements, reveals the transition of thermal-expansion-induced interface straining of nanotubes, which shifts from compression to tension inside the ceramic matrix under thermal loadings. This study provides valuable insights into the thermomechanical behavior of BNNT-reinforced ceramic nanocomposites and contributes to the optimal design of ceramic materials with tunable and zero CTE.