We study the evolution of the scaling relations that compare the effective density ( r r , e e S < ) and core density ( r , 1 1 S < kpc) to the stellar masses of star-forming galaxies (SFGs) and quiescent galaxies. These relations have been fully in place since z 3 and have exhibited almost constant slope and scatter since that time. For SFGs, the zero points in e S and 1 S decline by only 2 . This fact plus the narrowness of the relations suggests that galaxies could evolve roughly along the scaling relations. Quiescent galaxies follow different scaling relations that are offset to higher densities at the same mass and redshift. Furthermore, the zero point of their core density has declined by only 2 since z 3 , while the zero point of the effective density declines by 10 . When galaxies quench, they move from the star-forming relations to the quiescent relations. This involves an increase in the core and effective densities, which suggests that SFGs could experience a phase of significant core growth relative to the average evolution along the structural relations. The distribution of massive galaxies relative to the SFR-M and the quiescent M Srelations exhibits an L-shape that is independent of redshift. The knee of this relation consists of a subset of "compact" SFGs that are the most likely precursors of quiescent galaxies forming at later times. The compactness selection threshold in 1 S exhibits a small variation from z=3 to 0.5, M 0.65 log 10.5 9.6 9.3 1 * S --> -( ) M e kpc −2 , allowing the most efficient identification of compact SFGs and quiescent galaxies at every redshift.