Dogs are increasingly used in a wide range of detection tasks including explosives, narcotics, medical, and wildlife detection. Research on detection dog performance is important to understand olfactory capabilities, behavioral characteristics, improve training, expand deployment practices, and advance applied canine technologies. As such, it is important to understand the influence of specific variables on the quantification of detection dog performance such as test design, experimental controls, odor characteristics, and statistical analysis. Methods for testing canine scent detection vary influencing the outcome metrics of performance and the validity of results. Operators, management teams, policy makers, and law enforcement rely on scientific data to make decisions, design policies, and advance canine technologies. A lack of scientific information and standardized protocols in the detector dog industry adds difficulty and inaccuracies when making informed decisions about capability, vulnerability, and risk analysis. Therefore, the aim of this review is to highlight important methodological issues and expand on considerations for conducting scientifically valid detection dog research.