Background and purpose Defining refractory myasthenia gravis is important, as this can drive clinical decision making, for example, by escalating treatments in refractory individuals. There are several definitions of refractory myasthenia, and their performances have not been compared. Having valid and reliable criteria can help select patients in whom more aggressive treatments may be needed. Methods We applied five different refractory myasthenia criteria (Drachman, Mantegazza, Suh, the International Consensus Guideline (ICG), and the randomised controlled trial of eculizumab in refractory, anti‐acetylcholine receptor positive, generalised myasthenia gravis (REGAIN), to a cohort of 237 patients. We compared the proportion of refractory patients among different criteria and their scores on disease severity, fatigue, and quality‐of‐life (QoL) scales. We also assessed the agreement for each criterion between two trained assessors. Results The Drachman, Mantegazza, and Suh criteria resulted in high proportions of refractory individuals (40.1%, 39.2%, and 38.8%, respectively), compared with the ICG and REGAIN criteria (9.7% and 3.0%, respectively). Refractory patients by the ICG and REGAIN criteria had worse disease severity, QoL, and fatigue scores, compared with patients classified as refractory by other criteria. All criteria had high agreement between raters (between 70% and 100%). Conclusions There is high variability in the proportion of refractory myasthenia gravis patients depending on the criteria used, with ICG and REGAIN criteria capturing patients with worse disease severity. This reflects conceptual differences as to what refractory means. Further multicenter studies are needed to determine appropriate criteria for refractory myasthenia gravis.