Pulmonary hypertension (PH) is a progressive disease that arises from multiple etiologies and ultimately leads to right heart failure as the predominant cause of morbidity and mortality. In patients, distinct inflammatory responses are a prominent feature in different types of PH, and various immunomodulatory interventions have been shown to modulate disease development and progression in animal models. Specifically, PH-associated inflammation comprises infiltration of both innate and adaptive immune cells into the vascular wall of the pulmonary vasculature—specifically in pulmonary vascular lesions—as well as increased levels of cytokines and chemokines in circulating blood and in the perivascular tissue of pulmonary arteries (PAs). Previous studies suggest that altered hemodynamic forces cause lung endothelial dysfunction and, in turn, adherence of immune cells and release of inflammatory mediators, while the resulting perivascular inflammation, in turn, promotes vascular remodeling and the progression of PH. As such, a vicious cycle of endothelial activation, inflammation, and vascular remodeling may develop and drive the disease process. PA stiffening constitutes an emerging research area in PH, with relevance in PH diagnostics, prognostics, and as a therapeutic target. With respect to its prognostic value, PA stiffness rivals the well-established measurement of pulmonary vascular resistance as a predictor of disease outcome. Vascular remodeling of the arterial extracellular matrix (ECM) as well as vascular calcification, smooth muscle cell stiffening, vascular wall thickening, and tissue fibrosis contribute to PA stiffening. While associations between inflammation and vascular stiffening are well-established in systemic vascular diseases such as atherosclerosis or the vascular manifestations of systemic sclerosis, a similar connection between inflammatory processes and PA stiffening has so far not been addressed in the context of PH. In this review, we discuss potential links between inflammation and PA stiffening with a specific focus on vascular calcification and ECM remodeling in PH.