The changing social and policy context in which sport is produced, delivered, and consumed is considered as a foundation for proposing a new integrated framework that incorporates participation in sport with participation in leisure-time physical activity (PA) more broadly. In order to position sport in the broader context of leisure-time PA, the concept of physical literacy is reviewed and integrated into the theoretical foundations of a new lifespan framework. It is argued that historically, sport policy largely focused on competitive club-based sport and elite performance and that in line with this, talent development pathway models were developed and implemented. However, with increasing physical inactivity globally, these models do not apply to the general population. This is why we propose a population-based “whole of sport ecosystem” lifespan model—the Physical Activity and Sport Participation (PASP) framework. We conclude that this framework may serve as a holistic policy and implementation guide for all in the sport ecosystem. This includes governmental PA and sport policy makers, sport governing bodies and clubs, and the ever-increasing range of private PA and sport providers and also health agencies. In recognition of the changing patterns of participation in PA and sport across the lifespan, the PASP framework can contribute to coordinated and integrated PA and sport policy development, which, in turn, can lead to strategies that tackle the global physical inactivity crisis.