Even though deficits in olfactory function affect a considerable part of the population, the neuronal basis of olfactory deficits remains scarcely investigated. To achieve a better understanding of how smell loss affects neural activation patterns and functional networks, we set out to investigate patients with olfactory dysfunction using functional magnetic resonance imaging (fMRI) and olfactory stimulation. We used patients' scores on a standardized olfactory test as continuous measure of olfactory function. 48 patients (mean olfactory threshold discrimination identification (TDI) score = 16.33, SD = 6.4, range 6 - 28.5) were investigated. Overall, patients showed piriform cortex activation during odor stimulation compared to pure sniffing. Group independent component analysis indicated that the recruitment of three networks during odor stimulation was correlated with olfactory function: a sensory processing network (including regions such as insula, thalamus and piriform cortex), a cerebellar network and an occipital network. Interestingly, recruitment of these networks during pure sniffing was related to olfactory function as well. Our results support previous findings that sniffing alone can activate olfactory regions. Extending this, we found that the severity of olfactory deficits is related to the extent to which neural networks are recruited both during olfactory stimulation and pure sniffing. This indicates that olfactory deficits are not only reflected in changes in specific olfactory areas but also in the recruitment of occipital and cerebellar networks. These findings pave the way for future investigations on whether characteristics of these networks might be of use for the prediction of disease prognosis or of treatment success.