Milk is a beverage that completes human nutrition. It is produced by cows and goats and can be obtained by plants such as soy and coconut. The nutrition composition contained in kinds of milk is different from one another. The differences in nutrition composition have their identification potential, such as the processing, nutrition differences, purity, quality, etc. Hence, it is necessary to build a system that can identify milk types with a non-destructive method utilizing hyperspectral images and a Deep Learning algorithm. This research used a hyperspectral camera at a Visible and Near-Infrared (VNIR) range of light (400 -1000 nm). We used Convolutional Neural Network (CNN) as its image classification algorithm. Milk sample was collected from cow, goat, soy, and coconut and obtained exactly 1920 data. After the data was collected, we created datasets based on the type of classification tested. The category includes milk types with classes of animal-based and plant-based milk, the organisms that produce the milk with classes of coconut, cow, goat, and soy, and the processing method with classes of fresh and Ultra High Temperature (UHT). The tested algorithms of CNN architecture are GoogleNet, AlexNet, and Proposed CNN. The highest accuracy for 480 data was 100% reached by processing method classification of soy milk, and the computation took only 20 seconds. Meanwhile, the highest accuracy for 1920 data was 99.9% achieved by Proposed CNN architecture, and the calculation took only 78 seconds. These results showed that hyperspectral imaging and CNN algorithm are suitable for classifying types of milk.