In recent years, Dynamic Graph (DG) representations have been increasingly used for modeling dynamic systems due to their ability to integrate both topological and temporal information in a compact representation. Dynamic graphs efficiently handle applications such as social network prediction, recommender systems, traffic forecasting, or electroencephalography analysis, which cannot be addressed using standard numerical representations. As a direct consequence, dynamic graph learning has emerged as a new machine learning problem, combining challenges from both sequential/temporal data processing and static graph learning. In this research area, the Dynamic Graph Neural Network (DGNN) has become the state-of-the-art approach and a plethora of models have been proposed in the very recent years. This paper aims to provide a review of the problems and models related to dynamic graph learning. The various dynamic graph supervised learning settings are analyzed and discussed. We identify the similarities and differences between existing models concerning the way time information is modeled. Finally, we provide guidelines for DGNN design and optimization, and review public datasets for evaluating model performance on various tasks, along with the corresponding publications.