Comprehensive research is conducted on the design and control of the unmanned systems for electric vehicles. The environmental risk prediction and avoidance system is divided into the prediction part and the avoidance part. The prediction part is divided into environmental perception, environmental risk assessment, and risk prediction. In the avoidance part, according to the risk prediction results, a conservative driving strategy based on speed limit is adopted. Additionally, the core function is achieved through the target detection technology based on deep learning algorithm and the data conclusion based on deep learning method. Moreover, the location of bounding box is further optimized to improve the accuracy of SSD target detection method based on solving the problem of imbalanced sample categories. Software such as MATLAB and CarSim are applied in the system. Bleu-1 was 67.1, bleu-2 was 45.1, bleu-3 was 29.9 and bleu-4 was 21.1. Experiments were carried out on the database flickr30k by designing the algorithm. Bleu-1 was 72.3, bleu-2 was 51.8, bleu-3 was 37.1 and bleu-4 was 25.1. From the comparison results of the simulations of unmanned vehicles with or without a system, it can provide effective safety guarantee for unmanned driving.