The transmission of Escherichia coli (E. coli) bacteria to humans through infected fruits and vegetables, such as citrus, can lead to severe health issues, including bloody diarrhea and kidney disease (Hemolytic Uremic Syndrome). Therefore, the implementation of a suitable sensor and detection approach for inspecting the presence of E. coli colonies on fruits and vegetables would greatly enhance food safety measures. This article presents an evaluation of SafetySpect's Contamination, Sanitization Inspection, and Disinfection (CSI-D+) system, comprising an UV camera, an RGB camera, and illumination at fluorescence excitation wavelengths: ultraviolet C (UVC) at 275 nm. To conduct the study, eight different concentrations ranging from 100 (control) to 108 (maximum) cell counts of bacterial populations were inoculated on extracted citrus peel specimens. Specimen data could represent either irrigation or sprayer-based contamination events or direct contact with wildlife. Our study delves into early detection using the portable CSI-D+ system, capturing 240x240 pixel UV-C fluorescence images of E. Coli-inoculated grapefruit peel plugs. We developed a pipeline to prepare these images for the YOLOv8 deep learning framework, facilitating E. coli classification across varying concentrations and backgrounds. To enhance explainability, we employed Eigen Class Activation Map (Eigen-CAM) with YOLOv8, utilizing 'pytorch-eigen-cam' (https://github.com/rigvedrs/YOLO-V8-CAM) to elucidate the model's decision-making in detecting and classifying different E. coli concentrations. Our study demonstrated that the CSI-D+ system could classify fluorescence images at eight different concentration levels with an overall accuracy of more than 83% in which the control class reached a perfect classification accuracy while the images with E. coli concentration of 106 CFU/drop had the lowest accuracy of 71%. Similarly, the images with maximum concentration i.e., 108 CFU/drop were classified at an accuracy of 94%. These findings demonstrate the application of the CSI-D+ system as a rapid, non-invasive tool for E. coli detection on citrus peel surfaces that may be on the tree thus alerting the potential for similar contamination on fruit still on the tree. By providing timely insights, these results could enable effective intervention strategies to eliminate dangerous E. Coli from the food chain.