Objectives Chest X-rays (CXRs) convey much illegible physiological information that deep learning model (DLM) has been reported interpreting successfully. Since the electrocardiogram age established by DLM was revealed as a heart biological marker, we hypothesize that CXR age has similar potential to describe the heart and lung states. Therefore, we developed a DLM to predict sex and age through CXR and analyzed its relation with future cardiovascular diseases (CVD). Methods A total of 90,396 CXRs aged 20 to 90 were collected and separated into a development set with 53,102 CXRs and demographic information pairs, a tuning set with 7073 pairs, an internal validation set with 17,364 pairs, and an external validation set with 12,857 pairs. The study trained DLM with development set for estimating age and sex and compared them to actual information. Results The mean absolute errors of predicted age were 4.803 and 4.313 years in the internal and external validation sets, respectively. The area under the curve of sex analysis was 0.9993 and 0.9988 in the internal and external validation sets, respectively. Patients whose CXR age was 5 years older than chronologic age lead to higher risk of all-cause mortality (hazard ratio (HR): 2.42, 95% confidence interval (CI): 2.00–2.92), cardiovascular (CV)-cause mortality (HR: 7.57, 95% CI: 4.55–12.60), new-onset heart failure (HR: 2.07, 95% CI: 1.56–2.76), new-onset chronic kidney disease (HR: 1.73, 95% CI: 1.46–2.05), new-onset acute myocardial infarction (HR: 1.80, 95% CI: 1.12–2.92), new-onset stroke (HR: 1.45, 95% CI: 1.10–1.90), new-onset coronary artery disease (HR: 1.26, 95% CI: 1.04–1.52), and new-onset atrial fibrillation (HR: 1.43, 95% CI: 1.01–2.02). Conclusions Using DLM to predict CXR age provided additional information for future CVDs. Older CXR age is an accessible risk classification tool for clinician use.