Oil palm is a perennial plant that thrives well in tropical climate. Apart from humid environment, the plant also requires a wide variety of nutrients. Any deficiencies will directly affect its growth and palm oil production. These can often be detected from the change of leaf colour and texture. Deviations from the standard dark green colour indicates lack of certain nutrients. Therefore, this study proposes convolutional neural network (CNN) to classify nutrient deficiency in oil palms using leaf images. A total of 180 leaf images are acquired using standardized protocol. The samples are evenly distributed into healthy, nitrogen-deficient, and potassium-deficient groups. Residual network (ResNet)-50, visual geometry group-16 (VGG16), Densely connected network (DenseNet)-201, and AlexNet are trained and tested using the randomized samples. Each attained classification accuracies of 96.7%, 100%, 98.3%, and 100% respectively. Despite yielding similar performance, AlexNet is the more computational efficient architecture with less convolutional layers compared to VGG-16.