Citizen Science (CS) is a research approach that has become popular in recent years and offers innovative potential for dialect research in ornithology. As the scepticism about CS data is still widespread, we analysed the development of a 3-year CS project based on the song of the Common Nightingale (Luscinia megarhynchos) to share best practices and lessons learned. We focused on the data scope, individual engagement, spatial distribution and species misidentifications from recordings generated before (2018, 2019) and during the COVID-19 outbreak (2020) with a smartphone using the ‘Naturblick’ app. The number of nightingale song recordings and individual engagement increased steadily and peaked in the season during the pandemic. 13,991 nightingale song recordings were generated by anonymous (64%) and non-anonymous participants (36%). As the project developed, the spatial distribution of recordings expanded (from Berlin based to nationwide). The rates of species misidentifications were low, decreased in the course of the project (10–1%) and were mainly affected by vocal similarities with other bird species. This study further showed that community engagement and data quality were not directly affected by dissemination activities, but that the former was influenced by external factors and the latter benefited from the app. We conclude that CS projects using smartphone apps with an integrated pattern recognition algorithm are well suited to support bioacoustic research in ornithology. Based on our findings, we recommend setting up CS projects over the long term to build an engaged community which generates high data quality for robust scientific conclusions.