Analyzing and classifying five types of Lymphocyte White Blood Cell (WBC) is important to monitor the lack or excessive amount of cell in human body. These harmful amount of cell must be detected early for the early treatment can be run to the patient. However, the process may be tedious and time consuming as it is done manually by the experts. Other than that, it may yield inaccurate result as it depends on the pathologist skill and experience. This work presents a way that can be the second opinion to the experts using computer aided system as a solution. Convolutional Neural Network (CNN) is applied to the system to avoid complex structure and to eliminate the features extraction process. Three CNN models of mobilenet, resnet and VGG-16 is experimented on three different datasets which are kaggle, LISC and IDB-2. Kaggle, LISC and IDB-2 dataset consist of 6000, 242 and 260 images respectively. The result is divided into two parts which are dataset and model. As for IDB-2 dataset, the best model is VGG with training and validation accuracy of 0.9721 and 0.7913 respectively. While for kaggle and LISC dataset, the best model is resnet as it achieved training accuracy of 0.9713 and 0.9771 respectively. The highest validation accuracy for kaggle is 0.5955 and 0.5781 for LISC. Lastly, the best database that is most suitable for all model is IDB-2 database. It obtained highest training and validation accuracy for all model of mobilenet, resnet and VGG-16.