The pituitary gland has the primordial ability to dynamically adapt its cell composition to changing hormonal needs of the organism throughout life. During the first weeks after birth, an impressive growth and maturation phase is occurring in the gland during which the distinct hormonal cell populations expand. During pubertal growth and development, growth hormone (GH) levels need to peak which requires an adaptive enterprise in the GH-producing somatotrope population. At aging, pituitary function wanes which is associated with organismal decay including the somatopause in which GH levels drop. In addition to these key time points of life, the pituitary’s endocrine cell landscape plastically adapts during specific (patho-)physiological conditions such as lactation (need for PRL) and stress (engagement of ACTH). Particular resilience is witnessed after physical injury in the (murine) gland, culminating in regeneration of destroyed cell populations. In many other tissues, adaptive and regenerative processes involve the local stem cells. Over the last 15 years, evidence has accumulated that the pituitary gland houses a resident stem cell compartment. Recent studies propose their involvement in at least some of the cell remodeling processes that occur in the postnatal pituitary but support is still fragmentary and not unequivocal. Many questions remain unsolved such as whether the stem cells are key players in the vivid neonatal growth phase and whether the decline in pituitary function at old age is associated with decreased stem cell fitness. Furthermore, the underlying molecular mechanisms of pituitary plasticity, in particular the stem cell-linked ones, are still largely unknown. Pituitary research heavily relies on transgenic in vivo mouse models. While having proven their value, answers to pituitary stem cell-focused questions may more diligently come from a novel powerful in vitro research model, termed organoids, which grow from pituitary stem cells and recapitulate stem cell phenotype and activation status. In this review, we describe pituitary plasticity conditions and summarize what is known on the involvement and phenotype of pituitary stem cells during these pituitary remodeling events.