The inverted pendulum system (IPS) is considered the milestone of many robotic-based industries. In this paper, a new variant of variable structure adaptive fuzzy (VSAF) is used with new reduced linear quadratic regulator (RLQR) and feedforward gain for enhancing the stability of IPS. The optimal determining of VSAF parameters as well as Q and R matrices of RLQR are obtained by using a modified grey wolf optimizer with adaptive constants property via particle swarm optimization technique (GWO/PSO-AC). A comparison between the hybrid GWO/PSO-AC and classical GWO/PSO based on multi-objective function is provided to justify the superiority of the proposed technique. The IPS equipped with the hybrid GWO/PSO-AC-based controllers has minimum settling time, rise time, undershoot, and overshoot results for the two system outputs (cart position and pendulum angle). The system is subjected to robustness tests to ensure that the system can cope with small as well as significant disturbances.