Mushrooms are one of the plants that have so many varieties. Every variety has a different shape and color. But most people still feel difficult to know and classify every mushroom. Therefore, classification for mushroom is needed. Method for this research are Hue Saturation Value (HSV) as color segmentation, then Histogram of Oriented Gradient (HOG) as feature extraction, and Support Vector Machine (SVM) as a classification method. Mushrooms that being use are Agaricus, Amanita, Boletus, Cortinarius, Entoloma, Hygrocybe, Lactarius, Russula, Suillus. Total of mushrooms for this research are 900, with 100 each genus. This research using the k-fold Cross Validation method for 4-fold. From 900 images there are 675 for the training phase and 225 for the testing phase. Overall for this research got precision, recall, accuracy respectively 23.80%, 22.94%, and 82.69%. The best mushroom was Boletus with precision, recall, accuracy respectively 55.37%, 46.84%, and 89.69%.