Disengagement of advanced driver-assist systems (ADAS) has significant safety implications in partially automated vehicles where drivers may not be paying attention; this requires rigorous testing to ensure vehicles can safely transfer control to drivers. Existing testing and regulation on this topic are minimal, and do not address the role of difficult-to-measure but potentially important variables relating to individual vehicle differences or variations in environmental conditions. This study assessed variability in takeover alerting performance in three Model 3 Tesla vehicles in a demanding driving setting on a closed track. Results revealed that once an alerting sequence had begun, the ADAS systems performed as designed, but the vehicles were highly variable in the timing of alert initiation following the loss of lane marking tracking. In addition, luminosity and sun angle were found to potentially have a role in alert initiation, but dashboard-mounted cameras may be insufficient for measuring variations in atmospheric brightness. We recommend that future work studies brightness, as well as more comprehensive sampling guidelines for vehicle test protocols to ensure differences are adequately considered.