Humans frequently interact with Pacific harbor seals (Phoca vitulina richardii) at Punta Banda Estuary, Baja California, Mexico, due to the high incidence of recreational activities people undertake there. The immediate effect of these interactions is that seals flush to the water, reducing their time on land and, probably, increasing their energy expenditure. On-land observations were used to study the impact of different sources of disturbance on seal behavior and evaluate their effect on the amount of time dedicated to nursing over three pupping seasons, (2015–2017), with 0.58–0.81 disturbance events/hour recorded over the entire sampling period. Terrestrial vehicles were the source with the highest disturbance rate (number of disturbance events/h), followed closely by pedestrians. However, the proportion of seals affected was highest when pedestrians were the disturbance source. Recovery events (seals hauling out after flushing) occurred after 34% of disturbance events, after less than half of which the same number of hauled-out seals as there were prior to the disturbance were observed. Recovery time varied among the years studied, of which 2017 saw the longest recovery time. In addition, pedestrians were the disturbance source with the longest recovery time. Given that resting on land is essential for pup survival, which depends on both the establishment of the mother-pup bond from birth and its maintenance throughout nursing, flushing behavior may have significant implications for the entire colony during the nursing season. We recorded a decrease in nursing duration, which did not return to the same level even after recovery and the resumption of nursing. Terrestrial vehicles were found to be the disturbance source that shortened nursing events most significantly.