Visual storytelling describes the communication of stories through illustrations, graphics, imagery, and video instead of, or in addition to, oral, written, and audio formats. Compared to their popularity and wide reach, empirical research on map-based visual stories remains limited. We work towards infilling this gap through an empirical study on data journalism, providing the first assessment of four emerging design considerations for visual storytelling with maps: story map themes and their constituent narrative elements, visual storytelling genres, visual storytelling tropes, and individual audience differences. Specifically, we recruited 125 participants to an online map study, requiring them to separately review two visual stories and respond to a series of free-response and Likert scale questions regarding their retention, comprehension, and reaction. We followed a 2×2×2 factorial design for the visual stories, varying their themes (US presidential campaign donations, US coastal sea-level rise), genres (longform infographic, dynamic slideshow), and tropes (color highlighting, leader lines), while holding other design dimensions constant. The story theme did not influence the participants’ total retention or comprehension, indicating that a three-act narrative and its constituent elements can be applied consistently and effectively across variable kinds of topics. Instead, genres and, to a weaker degree, tropes influenced total participant retention, pointing to the importance of intentional design in map-based visual storytelling. Participants overall performed better when the visual storytelling designs used longform infographics or “scrollytelling” (genres) to structure content and leader lines (tropes) to visually accent information. In contrast, the story theme influenced audience reaction, with participants feeling significantly more concerned about and upset with the US presidential campaign donations story compared to the US sea-level rise story. Individual audience differences by expertise, motivation, and prior beliefs also influenced participant reaction. Our study signals a need for establishing a research and education agenda on map-based visual storytelling in both cartography and data journalism.