Large elastic arterial stiffening and endothelial dysfunction are phenotypic characteristics of vascular aging, a major risk factor for age-associated cardiovascular diseases. Compared to men, vascular aging in women appears to be slowed until menopause, whereafter vascular aging accelerates to match that seen in men. These sex differences in vascular aging have been attributed to changes in sex hormones that occur with aging. Although the role of estradiol in vascular aging in women has been highlighted in recent aging research, little is known about the impact of declining testosterone concentrations in both sexes. Importantly, while androgen concentrations generally decline with age in men, there are data that indicate reductions in androgen concentrations in women as well. Evidence suggests that low testosterone is associated with impaired endothelial function and increased arterial stiffness in men, although the effect of androgens on vascular aging in women remains unclear. Testosterone may modulate vascular aging by mitigating the effects of oxidative stress and inflammation, although there is sex specificity to this effect. The purpose of this review is to present and summarize the research regarding sex differences in vascular aging in response to androgens, specifically testosterone. Because exercise is a potent lifestyle factor for slowing and reversing vascular aging, we briefly summarize the available literature regarding the regulatory function of testosterone on vascular adaptations to exercise training.