The civil security sirens are used by the authorities in a wide range of countries to signal an imminent or ongoing threat. Even if their sound level is known, it is nevertheless difficult to evaluate their audibility across a given zone, especially in complex urban environments. An experimental protocol was deployed around a siren installed in a town in France, to assess its audibility perceptually and through modeling. Sound level measurements during source activation were made with the NoiseCapture smartphone application at different distances and on several axes by a group of 25 participants. They were also asked to fill in a questionnaire on perceptual information about the siren such as its audibility, the perceived sound level, or the masking of the siren by passing vehicles. A comparison between acoustic measurement levels using NoiseCapture and simulated sound levels using NoiseModelling was performed. The results of this study validate the use of the Common Noise Assessment Methods in Europe model to evaluate the audibility of a warning system located in an urban environment within a radius of 2.8 km around the siren. Finally, a metric linking audibility to modeled sound level is proposed, enabling the development of siren audibility maps in the study area.