This paper presents the stability improvement of the three-phase four-wire (3P-4W) grid-tied PV-hybrid energy storage system (HESS) using chaotic grew wolf optimization (CGWO) for DC bus voltage and AC bus voltage control. The CGWO tuned fractional order proportional–integral (FOPI) controllers reduce the and variations during diverse, dynamic conditions, i.e., sudden irradiation variations, deep voltage sag/swell, etc. The DC bus is responsible for the current injection/extraction control, maximum PV power extraction, bi-directional power flow, dc second-harmonics component elimination, and active power balance. At the point of common coupling (PCC), the AC bus is accountable for bi-directional power flow and active and reactive power control. The two-level voltage source converter (VSC) is controlled by a novel variable step-size incremental least mean square (VSS-ILMS) in zero voltage regulation (ZVR) mode. Due to its varying step size, VSC control is less prone to noise signals offers better stability, improved convergence rate, dc offset rejection, and tracking speed during dynamics, i.e., large oscillations. A battery and ultracapacitor are coupled to the DC link by buck-boost converters in the HESS. To regulate power transit between the DC bus and the grid, the HESS current control technique is designed to shift frequently from charging to discharging stage and vice versa. The novelty of the PV-HESS system lies in CGWO tuned VSS-ILMS control of VSC, which effectively and efficiently filter out the active fundamental constituents of load current and eliminate dc offset from VSC output. The HESS control maintains the DC bus voltage profile by absorbing and delivering energy (during dynamic conditions) rather than curtailing it. The presented system is simulated in a MATLAB/SIMULINK environment. The simulation results in graphical and numerical forms verify the stable and satisfactory operation of the proposed system as per IEEE519 standard.