Driver identification systems that use sequential models based on deep neural networks have been studied for personalized intelligent vehicles. After a vehicle starts moving for a trip, the system identifies the driver at each time step using the accumulated driving sensing data. We propose a novel driver identification system with temporal early exiting to identify a driver as early as possible while maintaining accuracy. Existing systems require entire-trip data or fixed-length partial trip data, regardless of the difficulty of driver identification. The proposed system automatically identifies the driver with less driving data for easy-to-identify trips and more driving data for hard-to-identify trips. To adaptively exit the identification by considering the difficulty of a trip, we propose a temporal early-exiting method by thresholding the confidence score. Sequential models output an identified driver and confidence score at each time step. However, the confidence score of deep neural networks is unreliable owing to the overconfidence problem. To overcome this, we propose three temporal confidence calibration methods that adjust the calibration strength according to the driving time and difficulty of the trip. Thus, the system can determine the best time to exit the identification, considering the trade-off between the latency and accuracy. Our experiments with a naturalistic driving dataset show that the proposed system achieved 90.06% accuracy with exiting early at an average of 6.7 min, yielding the same accuracy and achieving 74.2% latency reduction compared with driver identification with 26 min of fixed-length data for each trip.