The chaos oscillation particle swarm optimization (COPSO) algorithm is prone to binge trapped in the local optima when dealing with certain complex models in ground-penetrating radar (GPR) data inversion, because it inherently suffers from premature convergence, high computational costs, and extremely slow convergence times, especially in the middle and later periods of iterative inversion. Considering that the bilateral connections between different particle positions can improve both the algorithmic searching efficiency and the convergence performance, we first develop a fast single-trace-based approach to construct an initial model for 2-D PSO inversion and then propose a TV-regularization-based improved PSO (TVIPSO) algorithm that employs total variation (TV) regularization as a constraint technique to adaptively update the positions of particles. B by adding the new velocity variations and optimal step size matrices, the search range of the random particles in the solution space can be significantly reduced, meaning blindness in the search process can be avoided. By introducing constraint-oriented regularization to allow the optimization search to move out of the inaccurate region, the premature convergence and blurring problems can be mitigated to further guarantee the inversion accuracy and efficiency. We report on three inversion experiments involving multilayered, fluctuated terrain models and a typical complicated inner-interface model to demonstrate the performance of the proposed algorithm. The results of the fluctuated terrain model show that compared with the COPSO algorithm, the fitness error (MAE) of the TVIPSO algorithm is reduced from 2.3715 to 1.0921, while for the complicated inner-interface model the fitness error (MARE) of the TVIPSO algorithm is reduced from 1.9539 to 1.5674.