Cervical cancer is one of the most common cancers among women in the world, over 570,000 patients are affected annually. Pathological examination for patients using Pap Smear becomes the mainstream of cervical cancer diagnoses. Accurate diagnoses and analyses largely rely on 3 factors: cell segmentation, feature extraction and selection as well as classification. Firstly, a 2-layer segmentation algorithm based on block Maximum Between-Class Variance (Otsu) and Gradient Vector Flow (GVF) Snake model is applied to obtain regions of interest (ROI). Then the features of chroma, shape and texture are extracted and selected for a better classification performance. The random forest algorithm based on Artificial Fish Swarms Algorithm (AFSA) is used to recognize and classify cervical epithelial cells. The proposed methods were tested on 200 cervical Pap Smear images. Experimental results show that cervical cells can be segmented with an effective segmentation result. The proposed feature selection method achieved an accuracy of 81.31% with the minimum feature number. The improved random forest algorithm with 2 and 7 classification under fivefold cross-validation reaches the highest classification accuracy (96.86%). Experimental results showed that the proposed method has obvious recognition advantages, and thus provides a practical classification frame for the diagnoses of cervical cancer. INDEX TERMS Artificial fish swarm algorithm, random forest, cell detection, image segmentation.