Measuring the human perception of urban street space and exploring the street space elements that influence this perception have always interested geographic information and urban planning fields. However, most traditional efforts to investigate urban street perception are based on manual, usually time-consuming, inefficient, and subjective judgments. This shortcoming has a crucial impact on large-scale street spatial analyses. Fortunately, in recent years, deep learning models have gained robust element extraction capabilities for images and achieved very competitive results in semantic segmentation. In this paper, we propose a Street View imagery (SVI)-driven deep learning approach to automatically measure six perceptions of large-scale urban areas, including “safety”, “lively”, “beautiful”, “wealthy”, “depressing”, and “boring”. The model was trained on millions of people’s ratings of SVIs with a high accuracy. First, this paper maps the distribution of the six human perceptions of urban street spaces within the third ring road of Wuhan (appearing as Wuhan later). Secondly, we constructed a multiple linear regression model of “street constituents–human perception” by segmenting the common urban constituents from the SVIs. Finally, we analyzed various objects positively or negatively correlated with the six perceptual indicators based on the multiple linear regression model. The experiments elucidated the subtle weighting relationships between elements in different street spaces and the perceptual dimensions they affect, helping to identify the visual factors that may cause perceptions of an area to be involved. The findings suggested that motorized vehicles such as “cars” and “trucks” can negatively affect people’s perceptions of “safety”, which is different from previous studies. We also examined the influence of the relationships between perceptions, such as “safety” and “wealthy”. Finally, we discussed the “perceptual bias” issue in cities. The findings enhance the understanding of researchers and city managers of the psychological and cognitive processes behind human–street interactions.