The extracellular matrix (ECM) plays important roles in maintaining physiological structure and functions of various tissues and organs. Cardiac fibrosis is the excess deposition of ECM, including both fibrillar (collagens I and III) and non-fibrillar proteins. Characteristics of fibrosis can vary depending on the pathology, with focal fibrosis occurring following myocardial infarction (MI), and diffuse interstitial and perivascular fibrosis mainly in non-ischemic heart diseases. Compliance of the fibrotic tissue is significantly lower than the normal myocardium, and this can compromise the diastolic, as well as systolic dysfunction. Therefore, strategies to combat cardiac fibrosis have been investigated. Upon injury or inflammation, activated cardiac fibroblasts (myofibroblasts) produce more ECM proteins and cause fibrosis. The activation could be inhibited or the myofibroblasts could be ablated by targeting their specific expressed proteins. Modulation of tissue inhibitors of metalloproteinases (TIMPs) and moderate exercise can also suppress cardiac fibrosis. More recently, sex differences in cardiac fibrosis have come to light with differential fibrotic response in heart diseases as well as in fibroblast functions in vitro. This mini-review discusses recent progress in cardiac fibroblasts, TIMPs, sex differences and exercise in modulation of cardiac fibrosis.