Accurately phenotyping numerous test subjects is essential for most experimental research. Collecting such data can be tedious or time-consuming, and can be biased or limited by manual observations. The thermal tolerance of small ectotherms is a good example of this type of phenotypic data, and it is widely used to investigate thermal adaptation, acclimation capacity and climate change resilience of small ectotherms. Here, we present the results of automatically generated thermal tolerance data using motion tracking on video recordings using two Drosophila species and temperature acclimation to create variation in thermal tolerances and two different heat tolerance assays. We find similar effect sizes of acclimation and hardening responses between manual and automated approaches, but different absolute tolerance estimates. This discrepancy likely reflects both technical differences and the behavioral cessation of movement rather than physiological failure measured in other assays. We conclude that both methods generate biological meaningful results, which reflect different aspects of the thermal biology, find no evidence of inflated variance in the manually scored assays, but find that automation can increase throughput without compromising quality. Further we show that the method can be applied to a wide range of arthropod taxa. We suggest that our automated method is a useful example of through-put phenotyping, and suggest this approach might be applied to other tedious laboratory traits, such as desiccation or starvation tolerance, with similar benefits to through-put. However, the interpretation and potential comparison to results using different methodology rely on thorough validation of the assay and the involved biological mechanism.