The traffic accidents of the aged are increasing because old people generally experience a decline of physical functions and judgment but their dependence on automobiles increases. In order to investigate the older drivers' response to traffic signs and surrounding environment according to changing driving brightness, this study measured the visual cognitive behavior, driving behavior, and subjective evaluation while the driving brightness was changed using a driving simulator. Furthermore, the drivers who recognized traffic signs and those who did not were compared. As a result, it was found that some older drivers had declined ability of cognition, some failed to adapt to rapid brightness change even if sufficient attention was given to traffic sign, and some could not safely control the vehicle when approaching intersections. Therefore, the development of support systems for aged drivers in traffic environment is necessary in the future.