The understanding of stellar structure represents the crossroads of our theories of the nuclear force and the gravitational interaction under the most extreme conditions observably accessible. It provides a powerful probe of General Relativity on its strong field regime, and opens fruitful avenues for the exploration of new gravitational physics. The latter can be captured via the so-called modified theories of gravity, which modify the Einstein-Hilbert action of General Relativity and/or some of its building blocks/principles. These theories typically change the Tolman-Oppenheimer-Volkoff equations of stellar's hydrostatic equilibrium, having a large impact on the astrophysical properties of the corresponding stars, and thus opening a new window to constrain these theories with present and future observations of different types of stars. For relativistic stars, such as neutron stars, the uncertainty on the equation of state of matter at supranuclear densities intertwines with the new parameters coming from the modified gravity side, providing a whole new phenomenology for the typical predictions of stellar structure models, such as mass-radius relations, maximum masses, or moment of inertia. For non-relativistic stars, such as white, brown and red dwarfs, the weakening/strengthening of the gravitational force inside astrophysical bodies via the modified Newtonian (Poisson) equation may induce changes on the star's mass, radius, central density or luminosity, having an impact, for instance, in the Chandrasekhar's limit for white dwarfs, or in the minimum mass for stable hydrogen burning for high-mass brown dwarfs. This work aims to provide a broad overview of the main such results achieved in the recent literature for many such modified theories of gravity, by combining the results and constraints obtained from the analysis of relativistic and non-relativistic stars in different scenarios. Moreover, we will build a bridge between the efforts of the community working on different theories, formulations, types of stars, theoretical modellings, and observational aspects, highlighting some of the most promising opportunities in the field.