In recent years, with the rapid advance of computer science, various modern intelligent algorithms have successively emerged. Transformer, based on multi-head attention mechanism, is one of the most favored AI models among in this century. The introduction of these algorithms leads to dramatic progress in retrosynthesis prediction. Unlike conventional retrosynthesis prediction models, retrosynthesis prediction based on intelligent algorithms can automatically extract chemistry knowledge from chemical reaction datasets to predict retrosynthesis routes. In this review, we provide a comprehensive overview of retrosynthesis prediction based on modern intelligent algorithms, particularly artificial intelligence algorithm. After introducing the related deep learning model, the existing chemical reaction datasets and molecular representations are presented. Subsequently, the current state-of-the art of AI-assisted retrosynthesis prediction models in recent years is discussed, including template-based models, template-free models, and semi-template-based models. Additionally, we conclude by comparing retrosynthesis prediction models across different categorizations. Finally, several challenges and limitations of these current methods are summarized, with a view to promising directions for future research.