Human cytomegalovirus (CMV) is a highly infectious virus that is prevalent among adults worldwide. In immunocompetent individuals, CMV infection rarely causes complications. Following initial infection, however, CMV remains latent in the body and can be reactivated especially in immunocompromised patients, such as those with autoimmune inflammatory rheumatic diseases (AIIRDs), leading to tissue-invasive, life-threatening diseases, such as colitis, vascular thrombosis, and pneumonitis. Although there are standard treatment recommendations for symptomatic CMV disease, there is paucity of consensus in the role of primary and secondary antiviral prophylaxis to prevent CMV reactivation, especially in AIIRDs patients undergoing immunosuppressive therapy. Therein, this review aimed to provide an overview of the prevalence, clinical disease, risk factors, and treatment for CMV infection, as well as on the role of primary and secondary antiviral prophylaxis in AIIRDs patients. The emerging novel techniques that measure CMV-specific T-cell immunity as a guide to initiate prophylaxis in high-risk patients were also discussed.