Deep learning has gained a lot of research interest in artificial intelligence (AI) in many applications, such as image understanding, object detection, feature extraction, audio/ video processing, image demosaicking and denoising, overhead views in industrial applications. In addition, exploitation of deep learning in the field of data science, particularly in big data analytics focuses on high-level feature extraction and abstraction as data representation based on the hierarchical learning process. Moreover, deep learning is also designed to tackle supervised learning problems for a wide variety of tasks. How to reliably solve unsupervised tasks with a similar degree of success is an important issue to address. Such studies are investigated based on the adoption of parallel computing, i.e., GPUs and CPUs clusters.Various algorithms are already applied to achieve the desired goals. For instance, convolutional neural network has established superior performance on large-scale image and video classification. The supervised learning (semi/weakly) methods have expressively enhanced the performance when only a small amount of annotated data is available. In addition, correlation analysis, transfer learning, multi-tasking have proven