Geoscientists now identify coal layers using conventional well logs. Coal layer identification is the main technical difficulty in coalbed methane exploration and development. This research uses advanced quantile–quantile plot, self-organizing maps (SOM), k-means clustering, t-distributed stochastic neighbor embedding (t-SNE) and qualitative log curve assessment through three wells (X4, X5, X6) in complex geological formation to distinguish coal from tight sand and shale. Also, we identify the reservoir rock typing (RRT), gas-bearing and non-gas bearing potential zones. Results showed gamma-ray and resistivity logs are not reliable tools for coal identification. Further, coal layers highlighted high acoustic (AC) and neutron porosity (CNL), low density (DEN), low photoelectric, and low porosity values as compared to tight sand and shale. While, tight sand highlighted 5–10% porosity values. The SOM and clustering assessment provided the evidence of good-quality RRT for tight sand facies, whereas other clusters related to shale and coal showed poor-quality RRT. A t-SNE algorithm accurately distinguished coal and was used to make CNL and DEN plot that showed the presence of low-rank bituminous coal rank in study area. The presented strategy through conventional logs shall provide help to comprehend coal-tight sand lithofacies units for future mining.