Understanding physiological and behavioral responses to energy imbalances is important for the management of overweight/obesity and undernutrition. Changes in body composition and physiological functions associated with energy imbalances provide the structural and functional context in which to consider psychological and behavioral responses. Compensatory changes in physiology and behavior are more pronounced in response to negative than positive energy balances. The physiological and psychological impact of weight loss (WL) occur on a continuum determined by (i) the degree of energy deficit (ED), (ii) its duration, (iii) body composition at the onset of the energy deficit, and (iv) the psychosocial environment in which it occurs. Therapeutic WL and famine/semistarvation both involve prolonged EDs, which are sometimes similar in magnitude. The key differences are that (i) the body mass index (BMI) of most famine victims is lower at the onset of the ED, (ii) therapeutic WL is intentional and (iii) famines are typically longer in duration (partly due to the voluntary nature of therapeutic WL and disengagement with WL interventions). The changes in psychological outcomes, motivation to eat, and energy intake in therapeutic WL are often modest (bearing in mind the nature of the measures used) and can be difficult to detect but are quantitatively significant over time. As WL progresses, these changes become more marked. It appears that extensive WL beyond 10%-20% in lean individuals has profound effects on body composition and physiological function. At this level of WL, there is a marked erosion of psychological functioning, which appears to run in parallel to WL. Psychological resources dwindle and become increasingly focused on alleviating escalating hunger and food seeking behavior. Functional changes in fat-free mass, characterized by catabolism of skeletal muscle and organs may be involved in the drive to eat associated with semistarvation. Higher levels of body fat mass may act as a buffer to protect fat-free mass, functional integrity and limit compensatory changes in energy balance behaviors. The increase in appetite that accompanies therapeutic WL appears to be very different to the intense and all-consuming drive to eat that occurs during prolonged semistarvation. The mechanisms may also differ but are not well understood, and longitudinal comparisons of the relationship between body structure, function, and behavior in response to differing EDs in those with higher and lower BMIs are currently lacking.