The physical environment represents the very stage upon which the eco-evolutionary play unfolds. How fluctuations in the environment affect fitness is thus central to demographic projections, selection predictions, life history analyses, and conservation of populations. Here we systematically compare and explain why different ‘architectures’ of fluctuations—both in how they are generated in nature and constructed theoretically—alter fitness. Modelling efforts have mostly characterized variable environments with stochasticity. However, stochasticity refers to a random process. Many fluctuating environments in nature are non-random, driven by geophysical forces that create feedbacks or cyclicality: for example, seasons, tides, day/night, and disturbances such as floods, fires, and hurricanes. We show that evolutionary fitness logλs of a simple structured population is sensitive to the nature of fluctuations, even if different fluctuations appear indistinguishable in long-run statistical distributions. Importantly, we uncover two mathematical explanations of why fitness depends on the architecture of environmental fluctuations— consecutiveness of deviations from the environmental mean, and Jensen’s Inequality acting on nonlinear biological parameters—both arguably relevant features in virtually all populations inhabiting variable environments. The fitness divergence we demonstrate and explain is a useful step towards developing evolutionary demographic theory for the prevalent categories of non-random environments.