Shallow water is a complex sound propagation medium, which is affected by the varying spatial–temporal ocean environment. Taking this complexity into account, the classical processing techniques of source localization and environmental inversion may be improved. In this work, a joint tracking approach for the moving source and environmental parameters of the range-dependent and time-evolving environment in shallow water is presented. The tracking scheme treats both the source parameters (e.g., source depth, range, and speed) and the environmental parameters (e.g., water column sound speed profile (SSP) and sediment parameters) at the source location as unknown variables that evolve as the source moves. To counter sample impoverishment and robustly characterize the evolution of the parameters, an improved particle filter (PF), which is an extension of the standard PF, is proposed. Two examples with simulated data in a slowly changing environment and experimental data collected during the ASIAEX experiment are utilized to demonstrate the effectiveness of the joint approach. The results show that we were able to track the source and environmental parameters simultaneously, and the uncertainties were evaluated in the form of time-evolving posterior probability densities (PPDs). The performance comparison confirms that the improved PF is superior to the standard PF, as it can reduce the parameter uncertainties. The tracking capabilities of the improved PF were verified with high accuracy in real-time source localization and well-estimated rapidly varying parameters. Moreover, the influence of different particle numbers on the improved PF tracking performance is also illustrated.