Using Hubble Space Telescope observations of 147 host galaxies of low-mass black holes (BHs), we systematically study the structures and scaling relations of these active galaxies. Our sample is selected to have central BHs with virial masses of ∼10 5 -10 6 M . The host galaxies have total I-band magnitudes of −23.2 < M I < −18.8 mag and bulge magnitudes of −22.9 < M I < −16.1 mag. Detailed bulge-disk-bar decompositions with GALFIT show that 93% of the galaxies have extended disks, 39% have bars, and 5% have no bulges at all at the limits of our observations. Based on the Sérsic index and bulge-to-total ratio, we conclude that the majority of the galaxies with disks are likely to contain pseudobulges and very few of these low-mass BHs live in classical bulges. The fundamental plane of our sample is offset from classical bulges and ellipticals in a way that is consistent with the scaling relations of pseudobulges. The sample has smaller velocity dispersion at fixed luminosity in the Faber-Jackson plane compared with classical bulges and elliptical galaxies. The galaxies without disks are structurally more similar to spheroidals than to classical bulges according to their positions in the fundamental plane, especially the Faber-Jackson projection. Overall, we suggest that BHs with mass 10 6 M live in galaxies that have evolved secularly over the majority of their history. A classical bulge is not a prerequisite to host a BH.