Observations using the Fermi Large Area Telescope (Fermi-LAT) have found a significant γ-ray excess surrounding the center of the Milky Way (GC). One possible interpretation of this excess invokes γ-ray emission from an undiscovered population of either young or recycled pulsars densely clustered throughout the inner kiloparsec of the Milky Way. While these systems, by construction, have individual fluxes that lie below the point source sensitivity of the Fermi-LAT, they may already be observed in multiwavelength observations. Notably the Australia Telescope National Facility (ATNF) catalog of radio pulsars includes 270 sources observed in the inner 10 • around the GC. We calculate the γ-ray emission observed from these 270 sources and obtain three key results: (1) point source searches in the GC region produce a plethora of highly significant γ-ray "hotspots", compared to searches far from the Galactic plane, (2) there is no statistical correlation between the positions of these γ-ray hotspots and the locations of ATNF pulsars, and (3) the spectrum of the most statistically significant γ-ray hotspots is substantially softer than the spectrum of the GC γ-ray excess. These results place strong constraints on models where young pulsars produce the majority of the γ-ray excess, and disfavors some models where milli-second pulsars produce the γ-ray excess.