Locally advanced or metastatic urothelial bladder cancer (a/m UBC) is currently treated using platinum-based combination chemotherapy. Immune checkpoint inhibitors (ICIs) are the preferred second-line treatment options for cisplatin-eligible a/m UBC patients and as first-line options in cisplatin-ineligible settings. However, the response rates for ICI monotherapy are modest (~20%), which necessitates the exploration of alternative strategies. Dysregulated activation of fibroblast growth factor receptor (FGFR) signaling enhances tumor proliferation, survival, invasion, angiogenesis, and immune evasion. The recent U.S. Food and Drug Administration approval of erdafitinib and the emergence of other potent and selective FGFR inhibitors (FGFRis) have shifted the treatment paradigm for patients with a/m UBC harboring actionable FGFR2 or FGFR3 genomic alterations, who often have a minimal-to-modest response to ICIs. FGFRi–ICI combinations are therefore worth exploring, and their preliminary response rates and safety profiles are promising. In the present review, we summarize the impact of altered FGFR signaling on a/m UBC tumor evolution, the clinical development of FGFRis, the rationale for FGFRi–ICI combinations, current trials, and prospective research directions.