Up until now, head-up displays (HUDs) have been installed in front of the driver’s seat to provide drivers with auxiliary information. It may be used in various surroundings, ranging from very dark to very bright environments, such as daylight. A suitable text-background lightness combination can improve the driver’s visual clarity and efficiency when identifying displayed information to raise driving safety. Although many kinds of HUDs are designed to improve visual clarity by adjusting brightness, few studies have investigated the influence of the text-background lightness combination of a HUD on visibility, especially as the lighting level of the driving condition changes dramatically. In the study, 60 observers, comprising 20 young, 20 middle-aged, and 20 older participants, evaluated the visual clarity of 20 text-background lightness combinations on a HUD using paired comparison methods under dark and daylight surroundings (i.e., 300, 1500, and 9000 lx conditions). As a result, the combination of white text on a black background and black text on a white background presented the most significant preference and the best visual clarity under dark and daylight surroundings, respectively, improving visual safety when driving.