BackgroundThe breast cancer microenvironment contributes to tumor progression and response to chemotherapy. Previously, we reported that increased stromal Type X collagen α1 (ColXα1) and low TILs correlated with poor pathologic response to neoadjuvant therapy in estrogen receptor and HER2-positive (ER+/HER2+) breast cancer. Here, we investigate the relationship of ColXα1 and long-term outcome of ER+/HER2+ breast cancer patients in an adjuvant setting.MethodsA total of 164 cases with at least 5-year follow-up were included. Immunohistochemistry for ColXα1 was performed on whole tumor sections. Associations between ColXα1expression, clinical pathological features, and outcomes were analyzed.ResultsColXα1 expression was directly proportional to the amount of tumor associated stroma (p = 0.024) and inversely proportional to TILs. Increased ColXα1 was significantly associated with shorter disease free survival and overall survival by univariate analysis. In multivariate analysis, OS was lower in ColXα1 expressing (HR = 2.1; 95% CI = 1.2–3.9) tumors of older patients (> = 58 years) (HR = 5.3; 95% CI = 1.7–17) with higher stage (HR = 2.6; 95% CI = 1.3–5.2). Similarly, DFS was lower in ColXα1 expressing (HR = 1.8; 95% CI = 1.6–5.7) tumors of older patients (HR = 3.2; 95% CI = 1.3–7.8) with higher stage (HR = 2.7; 95% CI = 1.6–5.7) and low TILs. In low PR+ tumors, higher ColXα1 expression was associated with poorer prognosis.ConclusionColXα1 expression is associated with poor disease free survival and overall survival in ER+/HER2+ breast cancer. This study provides further support for the prognostic utility of ColXα1 as a breast cancer associated stromal factor that predicts response to chemotherapy.Electronic supplementary materialThe online version of this article (10.1186/s12885-019-6134-y) contains supplementary material, which is available to authorized users.