Objectives: Despite improved care for rheumatoid arthritis (RA) patients, many still experience treatment failure with biologic disease-modifying antirheumatic drugs (bDMARDs) or targeted synthetic DMARDs [tsDMARDs; typically Janus kinase inhibitors (JAKi)], and eventually switch to other agents. We compared the efficacy of a second tumor necrosis factor inhibitor (TNFi) and non-TNF-targeted treatment as the second-line treatment in patients showing an insufficient response to the first TNFi. Methods: Patients were included if they had received at least one prescription for a TNFi, and at least one follow-up prescription for a second TNFi or non-TNF-targeted treatment after discontinuation of the first drug. In total, 209 patients were analyzed, including 69 with a second TNFi and 140 with a non-TNF-targeted treatment (106 non-TNFi biologics and 34 JAKi). Cox regression was used to estimate the hazard ratio (HR) for discontinuation. Results: The mean follow-up period after switching was 28.0 (range: 0–80) months and 24.4% of the 209 patients switched or discontinued the second drug. In multivariate Cox proportional hazard analysis, the non-TNF-targeted treatment group had a lower likelihood of discontinuing their treatment than the second TNFi group [HR = 0.326, 95% confidence interval (CI): 0.170–0.626, p = 0.001]. When analyzed separately, the risk of discontinuation was significantly lower in both the non-TNFi biologic (HR = 0.318, 95% CI: 0.160–0.633, p = 0.001) and JAKi (HR = 0.356, 95% CI: 0.129–0.980, p = 0.046) groups than in the second TNFi group. Conclusion: Our study supported switching to a non-TNF-targeted treatment instead of TNF cycling in patients with RA showing an inadequate response to initial TNFi.