SummaryNowadays, intelligent transportation systems pay a lot of attention to autonomous vehicles it is believed that an autonomous vehicle improves mobility, comfort, safety, and energy efficiency. Making decisions is essential for the development of autonomous vehicles since these algorithms must be able to manage dynamic and complex urban crossings. In this research an optimal deep BiLSTM‐GAN classifier to detect the movement of smart vehicles, initially the preprocessing stage is performed to decrease noise in the received data after that the essential regions are next be extracted in the region of interest (ROI) to make the right decision. The extracted data are forwarded to the GAN for road segmentation as well as the optimized deep BiLSTM classifier, which recognizes the traffic sign, simultaneously making it possible to do a modified Hough line‐based maneuver prediction using the segmented information from the roads. Finally, the GAN determines the lane, and the BiLSTM predicts the traffic sign. The K‐nearest neighbor (KNN)‐based autonomous vehicle movement controllers are used to make the decision based on the predicted traffic sign and information about the lane. The proposed HSO algorithm was developed as the outcome of the common fusion of hawk and swarm optimization. Based on lane detecting achievements, at training percentage (TP) 90, the accuracy is 91.75%, Peak signal‐to‐noise ratio (PSNR) is 64.84%, mean square error (MSE) is 28.78, and mean absolute error (MAE) is 20.20, respectively, similarly based on the traffic sign prediction achievements at TP 90, the accuracy is 93.71%, sensitivity is 95.15%, specificity is 93.91%, and MSE is 28.78%, respectively.