The climate is a complex, chaotic system with many degrees of freedom. Attaining a deeper level of understanding of climate dynamics is an urgent scientific challenge, given the evolving climate crisis. In statistical physics, many-particle systems are studied using Large Deviation Theory (LDT). A great potential exists for applying LDT to problems in geophysical fluid dynamics and climate science. In particular, LDT allows for understanding the properties of persistent deviations of climatic fields from long-term averages and for associating them to low-frequency, large-scale patterns. Additionally, LDT can be used in conjunction with rare event algorithms to explore rarely visited regions of the phase space. These applications are of key importance to improve our understanding of high-impact weather and climate events. Furthermore, LDT provides tools for evaluating the probability of noise-induced transitions between metastable climate states. This is, in turn, essential for understanding the global stability properties of the system. The goal of this review is manifold. First, we provide an introduction to LDT. We then present the existing literature. Finally, we propose possible lines of future investigations. We hope that this paper will prepare the ground for studies applying LDT to solve problems encountered in climate science and geophysical fluid dynamics.