Skin cancer is the most common cancer in the USA, and it is a leading cause of death worldwide. Every year, more than five million patients are newly diagnosed in the USA. The deadliest and most serious form of skin cancer is called melanoma. Skin cancer can affect anyone, regardless of skin color, race, gender, and age. The diagnosis of melanoma has been done by visual examination and manual techniques by skilled doctors. It is a time-consuming process and highly prone to error. The skin images captured by dermoscopy eliminate the surface reflection of skin and give a better visualization of deeper levels of the skin. However, the existence of many artifacts and noise such as hair, veins, and water residue make the lesion images very complex. Due to the complexity of images, the border detection, feature extraction, and classification process are challenging. Without a proper mechanism, it is hard to identify and predict melanoma at an early stage. Therefore, there is a need to provide precise details, identify early skin cancer, and classify skin cancer with appropriate sensitivity and precision. This article aims to review and analyze two deep neural network-based classification algorithms (convolutional neural network, CNN; recurrent neural network, RNN) and a decision tree-based algorithm (XG-Boost) on skin lesion images (ISIC dataset) and find which of these provides the best classification performance metric. Also, the performance of algorithms is compared using six different metrics—loss, accuracy, precision, recall, F1 score, and ROC.