A 2‐year‐old castrated male mixed breed dog presented to the North Carolina State Veterinary Teaching Hospital for chronic diarrhea with hematochezia and weight loss. Cytology performed on a rectal scraping revealed macrophages containing magenta, light pink, and variably blue granular inclusions, and phagocytosed material concerning for infectious organisms. Histopathology was consistent with granulomatous colitis and identified intra‐histiocytic bacterial organisms, confirmed by fluorescent in situ hybridization (FISH)—tissue culture‐confirmed Escherichia coli. Based on these findings, a diagnosis of granulomatous colitis was made. The patient was successfully treated with oral enrofloxacin, and near‐complete remission of signs was achieved within 6 weeks. This report describes a case of granulomatous colitis in a mixed breed dog, and is the first published description of the cytologic features of this uncommon disease, offering a valuable cytologic‐histologic correlation. In this case, the cytology was helpful in identifying features consistent with granulomatous colitis and prioritizing the differential diagnoses and diagnostic plan.