Kidney fibrosis is the common histological end-point of progressive, chronic kidney diseases (CKDs) regardless of the underlying etiology. The hallmark of renal fibrosis, similar to all other organs, is pathological deposition of extracellular matrix (ECM). Renal ECM is a complex network of collagens, elastin, and several glycoproteins and proteoglycans forming basal membranes and interstitial space. Several ECM functions beyond providing a scaffold and organ stability are being increasingly recognized, for example, in inflammation. ECM composition is determined by the function of each of the histological compartments of the kidney, that is, glomeruli, tubulo-interstitium, and vessels. Renal ECM is a dynamic structure undergoing remodeling, particularly during fibrosis. From a clinical perspective, ECM proteins are directly involved in several rare renal diseases and indirectly in CKD progression during renal fibrosis. ECM proteins could serve as specific non-invasive biomarkers of fibrosis and scaffolds in regenerative medicine. The gold standard and currently only specific means to measure renal fibrosis is renal biopsy, but new diagnostic approaches are appearing. Here, we discuss the localization, function, and remodeling of major renal ECM components in healthy and diseased, fibrotic kidneys and the potential use of ECM in diagnostics of renal fibrosis and in tissue engineering.