Titanium implants are the standard therapeutic option when restoring missing teeth and reconstructing fractured and/or diseased bone. However, in the 30 years since the advent of micro-rough surfaces, titanium’s ability to integrate with bone has not improved significantly. We developed a method to create a unique titanium surface with distinct roughness features at meso-, micro-, and nano-scales. We sought to determine the biological ability of the surface and optimize it for better osseointegration. Commercially pure titanium was acid-etched with sulfuric acid at different temperatures (120, 130, 140, and 150 °C). Although only the typical micro-scale compartmental structure was formed during acid-etching at 120 and 130 °C, meso-scale spikes (20–50 μm wide) and nano-scale polymorphic structures as well as micro-scale compartmental structures formed exclusively at 140 and 150 °C. The average surface roughness (Ra) of the three-scale rough surface was 6–12 times greater than that with micro-roughness only, and did not compromise the initial attachment and spreading of osteoblasts despite its considerably increased surface roughness. The new surface promoted osteoblast differentiation and in vivo osseointegration significantly; regression analysis between osteoconductivity and surface variables revealed these effects were highly correlated with the size and density of meso-scale spikes. The overall strength of osseointegration was the greatest when the acid-etching was performed at 140 °C. Thus, we demonstrated that our meso-, micro-, and nano-scale rough titanium surface generates substantially increased osteoconductive and osseointegrative ability over the well-established micro-rough titanium surface. This novel surface is expected to be utilized in dental and various types of orthopedic surgical implants, as well as titanium-based bone engineering scaffolds.