Aqueous pollen extracts of varying taxonomic relations were analyzed with surface enhanced Raman scattering (SERS) by using gold nanoparticles in aqueous suspensions as SERS substrate. This enables a selective vibrational characterization of the pollen water soluble fraction (mostly cellular components) devoid of the spectral contributions from the insoluble sporopollenin outer layer. The spectra of the pollen extracts are species-specific, and the chemical fingerprints can be exploited to achieve a classification that can distinguish between different species of the same genus. In the simple experimental procedure, several thousands of spectra per species are generated. Using an artificial neural network (ANN), it is demonstrated that analysis of the intrinsic biochemical information of the pollen cells in the SERS data enables the identification of pollen from different plant species at high accuracy. The ANN extracts the taxonomically-relevant information from the data in spite of high intra-species spectral variation caused by signal fluctuations and preparation specifics. The results show that SERS can be used for the reliable characterization and identification of pollen samples. They have implications for improved investigation of pollen physiology and for allergy warning.