We consider the problem of which distance-regular graphs with small valency are Cayley graphs. We determine the distance-regular Cayley graphs with valency at most 4, the Cayley graphs among the distance-regular graphs with known putative intersection arrays for valency 5, and the Cayley graphs among all distance-regular graphs with girth 3 and valency 6 or 7. We obtain that the incidence graphs of Desarguesian affine planes minus a parallel class of lines are Cayley graphs. We show that the incidence graphs of the known generalized hexagons are not Cayley graphs, and neither are some other distance-regular graphs that come from small generalized quadrangles or hexagons. Among some "exceptional" distance-regular graphs with small valency, we find that the Armanios-Wells graph and the Klein graph are Cayley graphs.