The visual ecology of nocturnal anurans is poorly understood, but there is growing evidence that vision plays a role in important behaviors such as mate choice. While several recent studies have demonstrated that females are responsive to visual cues when selecting mates, the forces responsible for these preferences are unknown. We investigated the responsiveness of female gray treefrogs, Hyla versicolor, to video playbacks of calling conspecific males in which we varied attributes of the vocal sac, a conspicuous visual characteristic of calling males and a target of female choice in other species. Females responded surprisingly strongly to the video playbacks, but did so indiscriminately with respect to variation in vocal sac characteristics. We followed up on these results with a series of additional tests that examined female responses to abstract stimuli. Females continued to respond to such stimuli, leading us to conclude that their behavior was related to a generalized phototactic response. Because of this, we were unable to make conclusions regarding female preferences for vocal sac characteristics. Nonetheless, our results are significant in two respects. First, we illustrate that despite much effort into improving video playback methodologies, challenges remain, and we offer our experimental design as a method to ensure that appropriate conclusions can be drawn from such studies. Second, we argue that the female phototactic response has potentially significant behavioral implications and in general the consequences of anuran visual preferences deserve further investigation.