Chemical vapor deposited (CVD) diamond films have a controversial history regarding their surface electronic properties. Hydrogenation is known to induce a p-type conductive surface layer, which is not present on non-hydrogenated samples. The enhanced surface conductance can decrease significantly after annealing under high vacuum conditions at as low as 200 C (a temperature which is sufficiently low to ensure that the hydrogen termination remains intact). Although the hydrogen is necessary for the surface conductance, the surface can be made poorly conductive without removing the hydrogen termination. We have performed scanning tunneling microscopy (STM) and scanning tunneling spectroscopy (STS) to better understand the origin of the enhanced surface conductance. Our STS experiments confirm that hydrogenation induces the appearance of a conductive surface layer, which can change considerably after high vacuum annealing. In this paper, we will discuss the conditions under which the surface conductivity can be restored. In particular we study the conductance changes during plasma hydrogenation and after exposing it to atmospheric conditions. Topographical STM scans confirm that the surface structure is not altered at low annealing temperatures.