The frailty syndrome is characterized by a decreased capacity to adequately respond to stressors. One of the most impaired physiological systems is the autonomous nervous system, which can be assessed through heart rate (HR) variability (HRV) analysis. In this article, we studied the chronotropic response (HR and HRV) to a walking test. We also analyzed HRV indices in rest as potential biomarkers of frailty. For this, a 160 m-walking test and two standing rest tests (before and after the walking) were performed by young (19–29 years old, n = 21, 57% women), middle-aged (30–59 years old, n = 16, 62% women), and frail older adults (>60 years old, n = 28, 40% women) and non-frail older adults (>60 years old, n = 15, 71% women), classified with the FRAIL scale and the Clinical Frailty Scale (CFS). Frequency domain parameters better allowed to distinguish between frail and non-frail older adults (low-frequency power LF, high-frequency power HF (nu), LF/HF ratio, and ECG-derived respiration rate EDR). Frail older adults showed an increased HF (nu) and EDR and a reduced LF (nu) and LF/HF compared to non-frail older adults. The increase in HF (nu) could be due to a higher breathing effort. Our results showed that a walk of 160 m is a sufficient cardiovascular stressor to exhibit an attenuated autonomic response in frail older adults. Several HRV indices showed to be potential biomarkers of frailty, being LF (nu) and the time required to reach the maximum HR the best candidates.