The glymphatic system is responsible for waste clearance in the brain. It is comprised of perivascular spaces (PVS) that surround penetrating blood vessels. These spaces are filled with cerebrospinal fluid and interstitial fluid, and can be seen with magnetic resonance imaging. Various algorithms have been developed to automatically label these spaces in MRI. This has enabled volumetric and morphological analyses of PVS in healthy and disease cohorts. However, there remain inconsistencies between PVS measures reported by different methods of automated segmentation. The present review emphasizes that importance of voxel-wise evaluation of model performance, mainly with the Sørensen Dice similarity coefficient. Conventional count correlations for model validation are inadequate if the goal is to assess volumetric or morphological measures of PVS. The downside of voxel-wise evaluation is that it requires manual segmentations that require large amounts of time to produce. One possible solution is to derive these semi-automatically. Additionally, recommendations are made to facilitate rigorous development and validation of automated PVS segmentation models. In the application of automated PVS segmentation tools, publication of image quality metrics, such as the contrast-to-noise ratio, alongside descriptive statistics of PVS volumes and counts will facilitate comparability between studies. Lastly, a head-to-head comparison between two algorithms, applied to two cohorts of astronauts reveals how results can differ substantially between techniques.