Pupillary unrest is an established indicator of drowsiness or sleepiness. How sympathetic and parasympathetic activity contribute to pupillary unrest is not entirely unclear. In this study, we investigated 83 young healthy volunteers to assess the relationship of pupillary unrest to other markers of the autonomic nervous system. Sample entropy (SE) and the established pupillary unrest index (PUI) were calculated to characterize pupil size variability. Autonomic indices were derived from heart rate, blood pressure, respiration, and skin conductance. Additionally, we assessed individual levels of calmness, vigilance, and mood. In an independent sample of 26 healthy participants, we stimulated the cardiovagal system by a deep breathing test. PUI was related to parasympathetic cardiac indices and sleepiness. A linear combination of vagal heart rate variability [root mean square of heart beat interval differences (RMSSD)] and skin conductance fluctuations (SCFs) was suited best to explain interindividual variance of PUI. Complexity of pupil diameter (PD) variations correlated to indices of sympathetic skin conductance. Furthermore, we found that spontaneous fluctuations of skin conductance are accompanied by increases of pupil size. In an independent sample, we were able to corroborate the relation of PUI with RMSSD and skin conductance. A slow breathing test enhanced RMSSD and PUI proportionally to each other, while complexity of PD dynamics decreased. Our data suggest that the slow PD oscillations (f < 0.15 Hz) quantified by PUI are related to the parasympathetic modulation. Sympathetic arousal as detected by SCFs is associated to transient pupil size increases that increase non-linear pupillary dynamics.