Within condensed matter, single fluorophores are sensitive probes of their chemical environments, but it is difficult to use their limited photon budget to image precisely their positions, 3D orientations, and rotational diffusion simultaneously. We demonstrate the polarized vortex point spread function (PSF) for measuring these parameters, including characterizing the anisotropy of a molecule's wobble, simultaneously from a single image. Even when imaging dim emitters (~500 photons detected), the polarized vortex PSF is able to obtain 12 nm localization precision, 4-8° orientation precision, and 26° wobble precision. We use the vortex PSF to measure the emission anisotropy of fluorescent beads, the wobble dynamics of Nile red (NR) within supported lipid bilayers, and the distinct orientation signatures of NR in contact with amyloid-beta fibrils, oligomers, and tangles. The unparalleled sensitivity of the vortex PSF transforms single-molecule microscopes into nanoscale orientation imaging spectrometers, where the orientations and wobbles of individual probes reveal structures and organization of soft matter that are nearly impossible to perceive using molecular positions alone.