In recent years, multiple Unmanned Aerial Vehicle (UAV) formation flight has attracted worldwide research interest, for its potential benefits of scalability and flexibility. In complex urban environments, the successful operation of those UAVs requires the system to provide certain safety level. As one of the key requirements, collision avoidance improves the system's ability to accommodate operational environment variations, and to perform multiple tasks. To achieve this, artificial potential field (APF) has been recognized as one of the most suitable methods along with drone control. Although there has been substantial relevant work on the APF for single UAV in static environment, more efforts are desired to address formation maneuvers in complex environments such as urban. Most importantly, the traditional APF algorithms do not account for random errors in navigation solutions, which can bring potential risk to the UAV system. In response, this paper proposes a new APF algorithm that employs navigation information in complex urban environments, and the goal is to realize UAV formation collision avoidance. By augmenting the APF algorithm with UAV navigation information, the potential risk caused by navigation uncertainty can be mitigated, especially in the Global Navigation Satellite System (GNSS) challenged environment. The principle of the new APF approach is adaptively estimating the parameters of potential field force function, using the variance of navigation information and user-defined confidence probability. This new approach is applied in the synchronized UAV formation collision avoidance control. As a result, the UAVs can achieve fast position and attitude adjustment with high safety confidence. To verify the algorithm, quadrotors with emulated GNSS receivers are used to generate observation data. These data are incorporated into a complex urban environment simulation, where multiple sets of virtual obstacles are injected. Results show that the proposed method can achieve safe and efficient collision avoidance for cooperative formation flight in urban GNSS challenged environment.