Staphylococcus caprae (S. caprae) is a catalase-positive, coagulase-negative organism that was first isolated from goat milk, and was later found to colonize healthy human skin, nails, and nasal mucosa. Rarely, this commensal organism can become pathogenic in humans. S. caprae has been implicated in a variety of human infections, with the highest incidence being in bone and joint infections. We describe a man who, after receiving facet joint injections for back pain, developed native vertebral discitis, vertebral osteomyelitis with phlegmon, and bilateral psoas abscesses, from which S. caprae was isolated.