Raman spectroscopy has been widely used for label-free biomolecular analysis of cell and tissue for pathological diagnosis in vitro and in vivo. AI technology facilitates disease diagnosis based on Raman spectroscopy including machine learning (PCA and SVM), manifold learning (UMAP) and deep learning (ResNet and AlexNet). However, it is not clear how to optimize the appropriate AI classification model for different types of Raman spectral data. Here, we selected five representative Raman spectral datasets, including endometrial carcinoma, hepatoma extracellular vesicles, bacteria, melanoma cell, diabetic skin, with different characteristics regarding sample size, spectral data size, Raman shift range, tissue sites, Kullback-Leibler (KL) divergence, and key Raman shifts, explore the performance of different AI models (e.g. PCA-SVM, SVM, UMAP-SVM, ResNet or AlexNet). Tissue sites mean that spectral collection sites from sample, KL divergence means the divergence between spectra of different types. We found that for dataset of large spectral data size, Resnet performed better than PCA-SVM and UMAP, for dataset of small spectral data size, PCA-SVM or UMAP performed better. We also optimized the network parameters (e.g. principal components, activation function, and loss function) of AI model based on data characteristics. Using AI classification models, the mean area under receiver operating characteristic curves (AUC) for representative datasets reached 0.966, with mean sensitivity of 89.6%, mean specificity of 95.4%, mean accuracy of 93.4%, and mean time expense of 5 seconds. By using data characteristic assisted AI classification model, the accuracy improves from 85.1% to 94.6% for endometrial carcinoma grading, from 77.l% to 90.7% for hepatoma extracellular vesicles detection, from 89.3% to 99.7% for melanoma cell detection, from 88.1% to 97.9% for bacterial identification, from 53.7% to 85.5% for diabetic skin screening. Furthermore, according to the saliency maps, we found classification-associated biomolecules (e.g. nucleic acid, tyrosine, tryptophan, cholesteryl ester, fatty acid, and collagen), which contribute to the pathological diagnosis classification. Data characteristic assisted AI classification model was demonstrated to improve the robustness and accuracy of Raman spectroscopy in pathological classification. Collectively, this study opens up new opportunities for accurate and rapid Raman optical biopsy.