This study investigates the impacts of road traffic conditions and driver’s characteristics on the takeover time in automated vehicles using a driving simulator. Automated vehicles are barely expected to maintain their fully automated driving capability at all times based on the current technologies, and the automated vehicle system transfers the vehicle control to a driver when the system can no longer be automatically operated. The takeover time is the duration from when the driver requested the vehicle control transition from the automated vehicle system to when the driver takes full control of the vehicle. This study assumes that the takeover time can vary according to the driver’s characteristics and the road traffic conditions; the assessment is undertaken with various participants having different characteristics in various traffic volume conditions and road geometry conditions. To this end, 25 km of the northbound road section between Osan Interchange and Dongtan Junction on Gyeongbu Expressway in Korea is modeled in the driving simulator; the experiment participants are asked to drive the vehicle and take a response following a certain triggering event in the virtual driving environment. The results showed that the level of service and road curvature do not affect the takeover time itself, but they significantly affect the stabilization time, that is, a duration for a driver to become stable and recover to a normal state. Furthermore, age affected the takeover time, indicating that aged drivers are likely to slowly respond to a certain takeover situation, compared to the younger drivers. With these findings, this study emphasizes the importance of having effective countermeasures and driver interface to monitor drivers in the automated vehicle system; therefore, an early and effective alarm system to alert drivers for the vehicle takeover can secure enough time for stable recovery to manual driving and ultimately to achieve safety during the takeover.