Leaf disease of plants causes great loss in productivity of crops. So proper take care of plants is mandatory. Plants can be affected by various diseases. So Early diagnosis of leaf disease is a good practice. Computer vision-based classification of leaf disease can be a great way in diagnosing diseases early. Early detection of diseases can lead to better treatment. Vision based technology can identify disease quickly. Though deep learning is trending and using vastly for recognition task, but it needs very large dataset and also consumes much time. This paper introduced a method to classify leaf diseases using Gist and LBP (Local Binary Pattern) feature. These manual feature extraction process need less time. Combination of gist and LBP features shows significant result in classification of leaf diseases. Gist is used as global feature and LBP as local feature. Gist can describe an image very well as a scene. LBP is robust to illumination changes and occlusions and computationally simple. Various diseases of different plants are considered in this study. Gist and LBP features from images are extracted separately. Images are pre-processed before feature extraction. Then both feature matrix is combined using concatenation method. Training and testing is done on different plants separately. Different machine learning model is applied on the feature vector. Result from different machine learning algorithms is also compared. SVM performs better in classifying plant's leaf dataset.