Estimating distribution and abundance of species depends on the probability at which individuals are detected. Butterflies are of conservation interest worldwide, but data collected with Pollard walks—the standard for national monitoring schemes—are often analyzed assuming that changes in detectability are negligible within recommended sampling criteria. The implications of this practice remain poorly understood. Here, we evaluated the effects of sampling conditions on butterfly counts from Pollard walks using the Arctic fritillary, a common but cryptic butterfly in boreal forests of Alberta, Canada. We used an open population binomial N‐mixture model to disentangle the effects of habitat suitability and phenology on abundance of Arctic fritillaries, and its detectability by sampling different conditions of temperature, wind, cloud cover, and hour of the day. Detectability varied by one order of magnitude within the criteria recommended for Pollard walks (P varying between 0.04 and 0.45), and simulations show how sampling in suboptimal conditions increases substantially the risk of false‐absence records (e.g., false‐absences are twice as likely than true‐presences when sampling 10 Arctic fritillaries at P = 0.04). Our results suggest that the risk of false‐absences is highest for species that are poorly detectable, low in abundance, and with short flight periods. Analysis with open population binomial N‐mixture models could improve estimates of abundance and distribution for rare species of conservation interest, while providing a powerful method for assessing butterfly phenology, abundance, and behavior using counts from Pollard walks, but require more intensive sampling than conventional monitoring schemes.