In subjects with peripheral vestibular disease and controls, we assessed: 1. The relationship between spatial anxiety and perceived stress, and 2. The combined contribution of spatial anxiety, spatial perspective-taking, and individual cofactors to dizziness-related handicap. 309 adults participated in the study (153 with and 156 without peripheral vestibular disease), including patients with bilateral vestibular deficiency, unilateral deficiency (evolution <3 or ≥3 months), Meniere’s disease, and Benign Paroxysmal Positional Vertigo. Assessments included: general health, personal habits, spatial anxiety (3-domains), perceived stress, spatial perspective-taking, dizziness-related handicap (3-domains), unsteadiness, sleep quality, motion sickness susceptibility, trait anxiety/depression, state anxiety, depersonalization/derealization. After bivariate analyses, analysis of covariance was performed (p ≤ 0.05). Spatial anxiety was related to unsteadiness and perceived stress, with an inverse relationship with trait anxiety (ANCoVA, adjusted R2 = 0.27–0.30, F = 17.945–20.086, p < 0.00001). Variability on perspective-taking was related to vestibular disease, trait and state anxiety, motion sickness susceptibility, and age (ANCoVA, adjusted R2 = 0.18, F = 5.834, p < 0.00001). All domains of spatial anxiety contributed to the Physical domain of dizziness-related handicap, while the Navigation domain contributed to the Functional domain of handicap. Handicap variability was also related to unsteadiness, spatial perspective-taking, quality of sleep, and trait anxiety/depression (ANCoVA, adjusted R2 = 0.66, F = 39.07, p < 0.00001). Spatial anxiety is related to perceived stress in adults both with and without vestibular disease, subjects with trait anxiety rated lower on spatial anxiety. State anxiety and acute stress could be helpful for recovery after peripheral vestibular lesion. Spatial anxiety and perspective-taking contribute to the Physical and Functional domains of dizziness-related handicap, possibly because it discourages behavior beneficial to adaptation.