In this work, we report the role of ice-like water adlayers (IWLs) formed under ambient conditions in between mechanically exfoliated as-prepared and patterned few layer graphene (FLG) and multi-layer graphene (MLG) on hydrophobic Si and hydrophilic SiO2/Si substrates. The growth of the IWL is probed by measuring the height changes in graphene using intermittent contact atomic force microscopy (IC-AFM) and their electrostatic effect is studied using electrostatic force microscopy (EFM) over time. It is found that more IWLs are formed within a shorter period of time, when both as-prepared graphene and underlying substrates are either hydrophobic or hydrophilic in nature. In contrast, AFM voltage nanolithographically patterned trenches on FLG and MLG on the Si substrate show quick formation of IWLs. The effect of IWL formed, on the dimensions of trenches, is correlated with the variation of the measured EFM phase shift over time. This study demonstrates the dependence of the formation of IWLs under ambient conditions on the affinity towards water, at the interface of graphene on hydrophobic and hydrophilic substrates, which has important implications for the performance of graphene-based nanoelectronic devices.