The ORCID identification number(s) for the author(s) of this article can be found under https://doi.org/10.1002/adma.202203449.Ternary hexagonal manganites, RMnO 3 (R = Sc, Y, In, and Dy-Lu), have been intensively studied as model system for improper ferroelectricity. [19] In contrast to proper ferroelectrics, The research on topological phenomena in ferroelectric materials has revolutionized the way people understand polar order. Intriguing examples are polar skyrmions, vortex/anti-vortex structures, and ferroelectric incommensurabilties, which promote emergent physical properties ranging from electricfield-controllable chirality to negative capacitance effects. Here, the impact of topologically protected vortices on the domain formation in improper ferroelectric ErMnO 3 polycrystals is studied, demonstrating inverted domain scaling behavior compared to classical ferroelectrics. It is observed that as the grain size increases, smaller domains are formed. Phase field simulations reveal that elastic strain fields drive the annihilation of vortex/anti-vortex pairs within the grains and individual vortices at the grain boundaries. The inversion of the domain scaling behavior has far-reaching implications, providing fundamentally new opportunities for topology-based domain engineering and the tuning of the electromechanical and dielectric performance of ferroelectrics in general.