Monitoring trends in wildlife communities is integral to making informed land management decisions and applying conservation strategies. Birds inhabit most niches in every environment and because of this they are widely accepted as an indicator species for environmental health. Traditionally, point counts are the common method to survey bird populations, however, passive acoustic monitoring approaches using autonomous recording units have been shown to be cost-effective alternatives to point count surveys. Advancements in automatic acoustic classification technologies, such as BirdNET, can aid in these efforts by quickly processing large volumes of acoustic recordings to identify bird species. While the utility of BirdNET has been demonstrated in several applications, there is little understanding of its effectiveness in surveying declining grassland birds. We conducted a study to evaluate the performance of BirdNET to survey grassland bird communities in Nebraska by comparing this automated approach to point count surveys. We deployed ten autonomous recording units from March through September 2022: five recorders in row-crop fields and five recorders in perennial grassland fields. During this study period, we visited each site three times to conduct point count surveys. We compared focal grassland bird species richness between point count surveys and the autonomous recording units at two different temporal scales and at six different confidence thresholds. Total species richness (focal and non-focal) for both methods was also compared at five different confidence thresholds using species accumulation curves. The results from this study demonstrate the usefulness of BirdNET at estimating long-term grassland bird species richness at default confidence scores, however, obtaining accurate abundance estimates for uncommon bird species may require validation with traditional methods.