The clinical management of bone defects caused by trauma or nonunion fractures remains a challenge in orthopedic practice due to the poor integration and biocompatibility properties of the scaffold or implant material. In the current work, the osteogenic properties of carboxyl-modified single-walled carbon nanotubes (COOH–SWCNTs) were investigated in vivo and in vitro. When human preosteoblasts and murine embryonic stem cells were cultured on coverslips sprayed with COOH–SWCNTs, accelerated osteogenic differentiation was manifested by increased expression of classical bone marker genes and an increase in the secretion of osteocalcin, in addition to prior mineralization of the extracellular matrix. These results predicated COOH–SWCNTs’ use to further promote osteogenic differentiation in vivo. In contrast, both cell lines had difficulties adhering to multi-walled carbon nanotube-based scaffolds, as shown by scanning electron microscopy. While a suspension of SWCNTs caused cytotoxicity in both cell lines at levels >20 μg/mL, these levels were never achieved by release from sprayed SWCNTs, warranting the approach taken. In vivo, human allografts formed by the combination of demineralized bone matrix or cartilage particles with SWCNTs were implanted into nude rats, and ectopic bone formation was analyzed. Histological analysis of both types of implants showed high permeability and pore connectivity of the carbon nanotube-soaked implants. Numerous vascularization channels appeared in the formed tissue, additional progenitor cells were recruited, and areas of de novo ossification were found 4 weeks post-implantation. Induction of the expression of bone-related genes and the presence of secreted osteopontin protein were also confirmed by quantitative polymerase chain reaction analysis and immunofluorescence, respectively. In summary, these results are in line with prior contributions that highlight the suitability of SWCNTs as scaffolds with high bone-inducing capabilities both in vitro and in vivo, confirming them as alternatives to current bone-repair therapies.