We report on a controllable, hybrid quantum system consisting of cold Rydberg atoms and an optical nanofiber interface. Using a two-photon ladder-type excitation in 87 Rb, we demonstrate both coherent and incoherent Rydberg excitation at submicron distances from the nanofiber surface. The 780-nm photon, near resonant to the 5S → 5P transition, is mediated by the cooling laser, while the 482-nm light, near resonant to the 5P → 29D transition, is mediated by the guided mode of the nanofiber. The population loss rate of the cold atom ensemble is used to measure the Rydberg population rate. A theoretical model is developed to interpret the results and link the population loss rate to the experimentally measured, effective Rabi frequency of the process. This work makes headway in the study of Rydberg atom-surface interactions at submicron distances and the use of cold Rydberg atoms for all-fibered quantum networks.