Design plays a major role in online advertising. While specific aspects such as color or animation have been extensively studied, there is a surprising lack of comprehensive research on the overall impact of visual design and aesthetics. This study delved into the effects of varying levels of visual complexity in banner ads using the eye-tracking method. Out of 108 participants browsing test webpages with specially designed banners, data from 90 adequately measured data sets were used in the study. Notably, ads with a low level of visual complexity outperformed ads with a high level of visual complexity. While users noticed complex ads slightly faster (by 0.84%), they fixated on them significantly less (by 9.09%) and looked at them less frequently (by 4.79%). An implemented survey questionnaire examining the user perception of the banners reinforced the superiority of simple ads, as they were perceived as 4.40% more appealing in comparison. The study further delved into the correlation between objectively and subjectively evaluated data, exploring the credibility of subjective methods in the process. Considering our results and findings from other studies, it was evident that visually complex ads demanded more cognitive effort, could be more distracting, negatively impacted attention, could contribute to banner blindness and were perceived as less appealing.