Background: The detection of impaired physical performance in older adults with cardiovascular disease is essential for clinical management and therapeutic decision-making. There is a requirement for an assessment tool that can be used conveniently, rapidly, and securely in clinical practice for screening decreased physical performance. Objective: The present study was performed to evaluate the association of office-based physical assessments with decreased physical performance and to compare the prognostic capability of these assessments in older adults with cardiovascular disease. Methods: A total of 1,040 patients aged 75 years and older with cardiovascular disease were included in this analysis. One-leg standing time (OLST) and handgrip strength were measured as office-based physical assessment tools, and short physical performance battery (SPPB), 6-min walk distance, and usual gait speed were also measured at hospital discharge as measurements of physical performance. All-cause mortality was assessed by death registry at the hospital. We examined the association of office-based measures with physical performance and all-cause mortality. Results: The areas under the curve of OLST for SPPB < 10, 6-min walk distance < 300 m, and usual gait speed < 1.0 m/s were 0.87 (95% CI 0.83–0.91), 0.83 (95% CI 0.80–0.86), and 0.81 (95% CI 0.78–0.85), respectively. The discrimination abilities of OLST for decreased physical performance were significantly higher than those of handgrip strength. After adjusting for the effects of patient characteristics, the hazard ratio for all-cause mortality in the < 3 s group for OLST was 1.68 (95% CI 1.06–2.67, p = 0.03). Handgrip strength, however, was not significantly associated with mortality risk in these participants. Conclusion: Short OLST, in particular < 3 s, is associated with decreased physical performance and elevated mortality risk in elderly patients with cardiovascular disease. OLST can be conveniently measured in the clinician’s office as a screening tool for impaired physical performance.