Coffee grading is the main procedure in producing homogenous local commercial fair system of pricing in the market and export. Grading coffee is a difficult task during the inspection, because it requires training and experience of the experts. In order to tackle grading difficulties in coffee producing industries and corporates have been employed and trained experts. Even if, those experts do not work effectively due to tiredness, costly, time consuming, inconsistency, bias and other factors. Digital image processing techniques based on automatically extracted features have been explored to classify Ethiopian coffee to corresponding quality grade labels. Samples of those coffee beans were taken from Yirgacheffe Coffee Farmers’ Cooperative Union. On average, 228 images were taken from each of three grade values or levels (grade 1, grade 2 and grade 3). The total number of images taken was 684 containing 6138 coffee beans. To extract coffee bean features and build a classification model for grading coffee, the state of art deep learning algorithm called convolutional Neural Network was used. Base on the experimental results classification accuracy obtained with testing coffee bean images for grade 1, grade 2 and grade 3 coffee beans was 99.51%, 97.56%, and 98.04%, respectively with the overall classification accuracy of 98.38%. This shows a promising result, even if, images are captured under the challenging condition without laboratory setup, such as illumination, different resolution, shadow and orientation which affects greatly the performance of the classifier and hence they are the future research direction that needs further investigations of noise removal techniques.