While Mount Vernon, Monticello, Montpelier, and Highland work to recover the lives of people enslaved by Washington, Jefferson, Madison, and Monroe, their institutional missions emphasize the importance of these four men within American history. The resulting impediments to honoring Black lives within these spaces can be best understood using the analytical framework of reputational politics and by recognizing the roles visitors have in reproducing the reputations of the presidents and the women and men they enslaved. We base our examination of visitors’ participation in these reputational politics on a systematic documentation of tours and exhibits combined with surveys of visitors. Our results suggest that there are significant differences among the four sites in how visitors balance the reputations of enslaved communities with those of the Founding Fathers. On the whole, the emphasis on the presidents’ important positions within American social memory continues to inhibit efforts to honor Black lives at presidential plantation museums.