A ubiquitous aspect of the human experience is the creation of and engagement with art. While the field of empirical aesthetics has traditionally focused on visual stimuli, researchers have more recently begun investigating the experience of many different artistic domains, such as architecture, dance, and poetry. However, most prior work has investigated a single artistic domain, rather than comparing across different categories of art. To better understand how aesthetic experiences may be similar (or different) across modalities, we sought to directly compare the features that contribute to aesthetic judgments of three categories of art: poetry, music, and paintings. Online participants either rated poems (N = 363), musical excerpts (N = 266), or images of paintings (N = 93) on four variables: vividness of evoked imagery, emotional arousal, emotional valence, and aesthetic appeal. Our results indicate that vividness was the most significant predictor of the aesthetic appeal of poetry, while emotional valence was a stronger predictor for music than the other two categories. Additionally, we found that music was less likely to be rated as highly arousing and also negatively valanced than the other two categories. Thus, while overall, vividness, valence, and arousal contribute to aesthetic appeal of poetry, music, and visual art, there are subtle differences between the three types of stimuli. These findings inform theories of aesthetic judgments, suggesting that such theoretical perspectives take into account potential differences in stimulus modality.