The problem of visual pollution is a growing concern in urban areas, characterized by intrusive visual elements that can lead to overstimulation and distraction, obstructing views and causing distractions for drivers. Large‐area advertising structures, such as billboards, while being effective advertisement mediums, are significant contributors to visual pollution. Illegally placed or huge billboards can also exacerbate those issues and pose safety hazards. Therefore, there is a pressing need for effective and efficient methods to identify and manage advertising structures in urban areas. This article proposes a deep‐learning‐based system for automatically detecting billboards using consumer‐grade unmanned aerial vehicles. Thanks to the geospatial information from the drone's sensors, the position of billboards can be estimated. Side by side with the system, we share the very first dataset for billboard detection from a drone view. It contains 1361 images supplemented with spatial metadata, together with 5210 annotations.