Developing a data-driven predictive biomarker with capacity to classify and characterize brain-based features benefits clinical diagnoses and treatment outcomes for smokers. Although deep learning (DL) has been used to predict biomarkers in massive medical imaging fields owing to its strong learning capability from unstructured and unlabeled data, no study has yet used such tactic with structural MRI data to identify or represent smoking-related brain characteristics. For the first time, this work presents a DL model (named contextual attention U-net, CAT) to identify brain characteristics of smokers from T1-weighted structural MRI data. CAT uses U-net as a backbone, in which a spatial attention module is embedded between the third and fourth downsampling sections, to overcome fails of convolution by drawing long-range contextual interactions, while four lightweight channel attention modules are catenated into skip connections, to improve the feature directivity and preserve local details. Experimental results demonstrate that CAT is able to not only more robustly uncover gray/white matter features of smokers, but also win better classification performance with regard to accuracy, precision, recall, and F-score measures, in comparison to stateof- the-art models. If compared with three classical machine learning-based algorithms, the average lift ratios (%) of CAT are 25.02, 25.72, 25.60, and 25.62 regarding each term. When compared with seven DL-based approaches, they are 18.46, 18.44, 18.91, and 19.06 (%), respectively. Therefore, CAT is able to recognize faint differences of gray/white matter between smokers and nonsmokers, and it has potentials to solve difficulties of biomarker estimation.