Heart sound auscultation is a physical examination routinely used in clinical practice to identify potential cardiac abnormalities. However, accurate interpretation of heart sounds requires specialized training and experience, thereby limiting its generalizability. Deep learning, a subset of machine learning, involves training artificial neural networks to learn from large datasets and perform complex tasks related to intricate patterns, such as disease diagnosis, event prediction, and clinical decision-making. Over the past decade, deep learning has been successfully applied to heart sound analysis with remarkable achievements. Meanwhile, as heart sound analysis is gaining attention, many public and private heart sound datasets have been established for model training. The massive accumulation of heart sound data improves the performance of deep learning-based heart sound models and extends their clinical application scenarios. In this review, we will compile the commonly used datasets in heart sound analysis, introduce the fundamentals and state-of-the-art techniques in heart sound analysis and deep learning, and summarize the current applications of deep learning for heart sound analysis and their limitations for future improvement.