Minerals are inorganic substances that formed homogeneously by the combination of chemicals that comprise the Earth’s crust and mines are the commercially valuable natural minerals. Since minerals form as a result of chemical interactions in different natural environments, each may have different physical properties. Manually classifying minerals with the naked eye results in distinguishing issues, economic losses, and performance problems. In this study, a deep learning–based hybrid method for distinguishing seven mineral types is proposed. Deep learning models are used to combine the features extracted from residual blocks. The most inefficient features in the feature set were chosen using metaheuristic optimization. The complement rule was then used to combine inefficient features from the feature set into clusters. Consequently, efficient features were obtained, with an overall classification accuracy of 96.14%. The use of the complement rule method in the data set, along with the optimization methods, was shown to improve classification performance. This study is expected to help improve the efficiency and speed of mineral classification.