Roads should deliver appropriate information to drivers and thus induce safer driving behavior. This concept is also known as “self-explaining roads” (SERs). Previous studies have demonstrated that understanding how road characteristics affect drivers’ speed choices is the key to SERs. Thus, in order to reduce traffic casualties via engineering methods, this study aimed to establish a speed decision model based on visual road information and to propose an innovative method of SER design. It was assumed that driving speed is determined by road geometry and modified by the environment. Lane fitting and image semantic segmentation techniques were used to extract road features. Field experiments were conducted in Tibet, China, and 1375 typical road scenarios were picked out. By controlling variables, the driving speed stimulated by each piece of information was evaluated. Prediction models for geometry-determined speed and environment-modified speed were built using the random forest algorithm and convolutional neural network. Results showed that the curvature of the right boundary in “near scene” and “middle scene”, and the density of roadside greenery and residences play an important role in regulating driving speed. The findings of this research could provide qualitative and quantitative suggestions for the optimization of road design that would guide drivers to choose more reasonable driving speeds.