<b><i>Introduction:</i></b> Fine-needle aspiration (FNA) is well-established for the evaluation of suspicious thyroid nodules. However, a significant proportion is nondiagnostic. Rapid on-site evaluation (ROSE) has been proposed to improve the overall adequacy of FNA. <b><i>Methods:</i></b> Retrospective cohort study comparing adequacy of thyroid FNA findings pre- and postimplementation of ROSE at a tertiary center in Switzerland. Patients undergoing thyroid FNA from January 2016 to December 2019 were included. The primary outcome was the rate of nondiagnostic findings (Bethesda System for Reporting Thyroid Cytopathology category I). <b><i>Results:</i></b> In total, 410 thyroid nodule FNAs were performed. Of those, 309 with standard FNA and 101 with ROSE. The majority of patients were female (71%), with a median age of 56 years (IQR 46–68) and a nodule diameter of 1.9 cm (IQR 1.2–2.9). Implementation of ROSE led to a decrease in nondiagnostic findings from 41.1% to 23.8%, with an odds ratio of 0.42 (95% CI: 0.24–0.72; <i>p</i> = 0.002). Implementation of ROSE was associated with significantly higher rates of Bethesda category III (27.7% vs. 19.1%), category IV (15.8% vs. 5.5%), and Bethesda category VI (6.9% vs. 2.3%). Repeated FNA was performed in 29.1% before and 20.8% after implementation of ROSE (<i>p</i> = 0.18). The mean number of FNA per nodule was reduced from 1.4 (0.6) to 1.2 (0.4) with ROSE (<i>p</i> = 0.04). <b><i>Conclusions:</i></b> Implementation of ROSE of thyroid nodule specimen improved diagnostic adequacy of FNA, reducing nondiagnostic findings. However, due to increased equivocal findings (Bethesda category III), there was no significant reduction of repeat FNA.