Infectious diseases caused by bacterial pathogens pose a significant public health threat, emphasizing the need for swift and accurate bacterial species detection methods. Hyperspectral microscopic imaging (HMI) offers nondestructive, rapid, and data‐rich advantages, making it a promising tool for microbial detection. In this research, we present a highly compatible and cost‐effective approach to extend a standard biomicroscope system into a hyperspectral biomicroscope using a prism‐grating‐prism configuration. Using this prototype, we generate 600 hyperspectral data cubes for Listeria, Bacillus typhi, Bacillus pestis, and Bacillus anthracis. Additionally, we propose a Transformer‐based classification network that achieves a 99.44% accuracy in classifying these infectious pathogens, outperforming traditional methods. Our results suggest that the successful combination of HMI and the optimized Transformer‐based classification network highlights the potential for rapid and precise detection of infectious disease pathogens .