Recent work has revealed that the light curves of hydrogen-poor (Type I) superluminous supernovae (SLSNe), thought to be powered by magnetar central engines, do not always follow the smooth decline predicted by a simple magnetar spin-down model. Here we present the first systematic study of the prevalence and properties of "bumps" in the post-peak light curves of 34 SLSNe. We find that the majority (44-76%) of events cannot be explained by a smooth magnetar model alone. We do not find any difference in the supernova properties between events with and without bumps. By fitting a simple Gaussian model to the light curve residuals, we characterize each bump with an amplitude, temperature, phase, and duration. We find that most bumps correspond with an increase in the photospheric temperature of the ejecta, although we do not see drastic changes in spectroscopic features during the bump. We also find a moderate correlation (ρ ≈ 0.5; p ≈ 0.01) between the phase of the bumps and the rise time, implying that such bumps tend to happen at a certain "evolutionary phase," (3.7 ± 1.4)t rise . Most bumps are consistent with having diffused from a central source of variable luminosity, although sources further out in the ejecta are not excluded. With this evidence, we explore whether the cause of these bumps is intrinsic to the supernova (e.g., a variable central engine) or extrinsic (e.g., circumstellar interaction). Both cases are plausible, requiring low-level variability in the magnetar input luminosity, small decreases in the ejecta opacity, or a thin circumstellar shell or disk.