The aim was to investigate the association between chronic low back pain (CLBP) and incident transient ischemic attack (TIA) and stroke in Germany. The present retrospective cohort study included adults aged ≥18 years who were diagnosed for the first time with CLBP in one of 1198 general practices in Germany in 2005–2019 (index date). Patients without CLBP were matched to those with CLBP (1:1) using a propensity score based on age, sex, the index year, the number of medical consultations per year during the follow-up, and the number of years of follow-up. In patients without CLBP, the index date was a randomly selected visit date. Both groups were followed for up to 10 years. There were 159,440 patients included in the study (mean (SD) age: 52.1 (16.5) years; 51.5% women). Within 10 years of the index date, 6.5% and 5.9% of patients with and without CLBP were diagnosed with TIA or stroke, respectively (log-rank p-value < 0.001). The Cox regression analysis corroborated these results, as there was a significant association between CLBP and incident TIA or stroke (HR = 1.28, 95% CI = 1.22–1.35). CLBP was positively and significantly associated with incident TIA and stroke in Germany. More research is warranted to better understand this relationship.