Infections of bone are severe complications, and an optimization of grafting material with antimicrobial drugs might be useful for prevention and treatment. This study aimed to investigate the influence of gentamicin-loaded bone graft on the healing of bone defects in a sheep model. Metaphyseal and diaphyseal drill hole defects (diameter: 6 mm, depth: 15 mm) were filled with graft or gentamicin-loaded graft (50 mg/g graft) or were left untreated. Analysis of regeneration after three and nine weeks, micro-computed tomography (μCT), and histology revealed a significant increase in bone formation in the drill hole defects, which began at the edges of the holes and grew over time into the defect center. The amount of graft decreased over time due to active resorption by osteoclasts, while osteoblasts formed new bone. No difference between the groups was seen after three weeks. After nine weeks, significantly less mineralized tissue was formed in the gentamicin-loaded graft group. Signs of inflammatory reactions were seen in all three groups. Even though the applied gentamicin concentration was based on the concentration of gentamicin mixed with cement, the healing process was impaired. When using local gentamicin, a dose-dependent, compromising effect on bone healing should be considered.