Background Despite common misconceptions, an individual may be seriously ill with a restrictive eating disorder without an outwardly recognizable physical sign of the illness. The aim of this qualitative study was to investigate the perspectives of individuals who have previously battled a restrictive eating disorder who were considered “not sick enough” by others (e.g., peers, families, healthcare professionals) at some point during their illness, and to understand the perceived impact on the illness and recovery. Such misconceptions are potentially damaging, and have been previously linked with delayed help-seeking and poorer clinical outcomes. Methods Seven women who had recovered from anorexia nervosa or atypical anorexia nervosa participated in semi-structured interviews. Interviews were transcribed and interpretive phenomenological analysis was used. Results Three main themes emerged: (1) dealing with the focus upon one’s physical appearance while battling a mental illness, (2) “project perfect”: feeling pressure to prove oneself, and (3) the importance of being seen and understood. Participants reported that their symptoms were occasionally met with trivialization or disbelief, leading to shame, confusion, despair, and for some, deterioration in eating disorder symptoms which drove further weight loss. In contrast, social support and being understood were viewed as essential for recovery. Conclusion To facilitate treatment seeking and engagement, and to optimize chances of recovery, greater awareness of diverse, non-stereotypical presentations of restrictive eating disorders is needed which challenge the myth that weight is the sole indicator of the presence or severity of illness.