Calcific aortic valve disease (CAVD) is a highly prevalent and progressive disorder that ultimately causes gradual narrowing of the left ventricular outflow orifice with ensuing devastating hemodynamic effects on the heart. Calcific mineral accumulation is the hallmark pathology defining this process; however, fibrotic extracellular matrix (ECM) remodeling that leads to extensive deposition of fibrous connective tissue and distortion of the valvular microarchitecture similarly has major biomechanical and functional consequences for heart valve function. Significant advances have been made to unravel the complex mechanisms that govern these active, cell-mediated processes, yet the interplay between fibrosis and calcification and the individual contribution to progressive extracellular matrix stiffening require further clarification. Specifically, we discuss (1) the valvular biomechanics and layered ECM composition, (2) patterns in the cellular contribution, temporal onset, and risk factors for valvular fibrosis, (3) imaging valvular fibrosis, (4) biomechanical implications of valvular fibrosis, and (5) molecular mechanisms promoting fibrotic tissue remodeling and the possibility of reverse remodeling. This review explores our current understanding of the cellular and molecular drivers of fibrogenesis and the pathophysiological role of fibrosis in CAVD.