To address the issues of local minima, target unreachability, and significant formation disruption during obstacle avoidance in the conventional artificial potential field (APF), a control approach that integrates APF with optimal consensus control which can achieve cooperative obstacle avoidance is proposed. Based on the double integrator multi-UAV formation model with a fixed undirected communication topology, the optimal consensus control protocol incorporating an obstacle avoidance cost function is introduced. This addresses the limitations of APF-based obstacle avoidance while simultaneously managing multi-UAV formation control. Training interactions in randomly generated unknown obstacle environments are conducted using Random Search for Hyperparameter Optimization (RSHO). Combined with the evaluation model, select the optimal solution of the consensus performance index, control consumption performance index, and obstacle avoidance performance index parameters of the multi-UAV formation control system. Furthermore, a virtual repulsive potential field is designed for each UAV to prevent inter-UAV collisions during obstacle avoidance. Simulation results show that the improved APF (IAPF) with optimal consensus control effectively overcomes the limitations of conventional APF. It achieves multi-UAV formation obstacle avoidance control in unknown environments and avoids the phenomenon of inter-UAV collisions during the obstacle avoidance process while maintaining formation integrity, accelerating formation reconfiguration and convergence, reducing consensus consumption and control loss due to obstacle avoidance, shortening mission time, and enhancing obstacle avoidance efficiency, highlighting the superiority of multi-UAV formation obstacle avoidance.