Bandgap reference (BGR) circuits play a crucial role as voltage reference generators for other components within a variety of analog integrated circuits. Therefore, their power supply rejection ratio (PSRR), which represents the BGR's ability to maintain a stable output in the presence of supply ripples, has a substantial impact on the overall circuit performance and is consequently worthy of attention. In this work, the design of a 65nm-CMOS BGR circuit and the computational approach to optimize its PSRR value are presented. Our proposed BGR circuit incorporates a modification in the op-amp structure to enhance the PSRR parameter. Detailed explanations of the proposed op-amp's differential gain calculation and the PSRR parameter formula are provided. Additionally, we employ the particle swarm optimization (PSO) algorithm to maximize PSRR while satisfying other specifications. Four distinct approaches for utilizing the inertia weight parameter of the PSO algorithm are explored: two newly developed techniques (PSO -local exploitation orienter (PSO-LEO) and PSO -global exploration orienter (PSO-GEO)) together with two conventional approaches. A comprehensive comparative analysis reveals the superiority of our proposed PSO-GEO technique. Ultimately, our 65 nm CMOS bandgap circuit exhibits exceptional performance, featuring a temperature coefficient of 6.4056 ppm/°C and a PSRR of 99.9896 dB at 1 kHz. In conclusion, this work contributes substantially through op-amp architecture modification, PSRR optimization via the PSO algorithm, and introducing new inertia weight parameter utilization strategies and analyzing them.