Surgery is the most commonly used method of curing inverted papilloma (IP) or nasal polyp (NP). Although accurate preoperative recognition by computed tomography (CT) is a critical aspect of surgical planning, the minor CT imaging differences in such lesions may be a challenge. Therefore, we have devised a deep learning framework for automatic recognition of IP and NP in CT. The proposed framework involves two major steps: (a) use of a convolutional neural network (CNN) to preclassify lesions and (b) automatic IP/NP recognition. The preclassify CNN enables classification of CT slices according to anatomic structure. Separate networks are then implemented to differentiate IP and NP accordingly. Once the framework was trained using a CT dataset (5681 slices) from 136 patients, it outperformed other methods during evaluation, achieving 89.30% accuracy (area under the curve [AUC]=0.95) in classification. The proposed framework has clear potential as a clinical tool, enabling effective and highly accurate preoperative recognition of IP and NP. INDEX TERMS Deep learning, inverted papilloma, nasal polyp, pre-classify, recognition.