Potatoes are one of the most demanded products due to their richness in nutrients. However, the lack of attention to external and, especially, internal defects greatly reduces its marketability and makes it prone to a variety of diseases. The present study aims to identify healthy-looking potatoes but with internal defects. A visible (Vis), near-infrared (NIR), and short-wavelength infrared (SWIR) spectrometer was used to capture spectral data from the samples. Using a hybrid of artificial neural networks (ANN) and the cultural algorithm (CA), the wavelengths of 861, 883, and 998 nm in Vis/NIR region, and 1539, 1858, and 1896 nm in the SWIR region were selected as optimal. Then, the samples were classified into either healthy or defective class using an ensemble method consisting of four classifiers, namely hybrid ANN and imperialist competitive algorithm (ANN-ICA), hybrid ANN and harmony search algorithm (ANN-HS), linear discriminant analysis (LDA), and k-nearest neighbors (KNN), combined with the majority voting (MV) rule. The performance of the classifier was assessed using only the selected wavelengths and using all the spectral data. The total correct classification rates using all the spectral data were 96.3% and 86.1% in SWIR and Vis/NIR ranges, respectively, and using the optimal wavelengths 94.1% and 83.4% in SWIR and Vis/NIR, respectively. The statistical tests revealed that there are no significant differences between these datasets. Interestingly, the best results were obtained using only LDA, achieving 97.7% accuracy for the selected wavelengths in the SWIR spectral range.