Background: Despite recent advancements, medical technology has not yet reached its peak. Precision medicine is growing rapidly, thanks to machine learning breakthroughs powered by increased computational capabilities. This article explores a deep learning application for computer-aided diagnosis in dermatology. Methods: Using a custom model based on EfficientNetB3 and deep learning, we propose an approach for skin lesion classification that offers superior results with smaller, cheaper, and faster inference times compared to other models. The skin images dataset used for this research includes 8222 files selected from the authors’ collection and the ISIC 2019 archive, covering six dermatological conditions. Results: The model achieved 95.4% validation accuracy in four categories—melanoma, basal cell carcinoma, benign keratosis-like lesions, and melanocytic nevi—using an average of 1600 images per category. Adding two categories with fewer images (about 700 each)—squamous cell carcinoma and actinic keratoses—reduced the validation accuracy to 88.8%. The model maintained accuracy on new clinical test images taken under the same conditions as the training dataset. Conclusions: The custom model demonstrated excellent performance on the diverse skin lesions dataset, with significant potential for further enhancements.