Polynomial chaos is a powerful technique for propagating uncertainty through ordinary and partial differential equations. Random variables are expanded in terms of orthogonal polynomials and differential equations are derived for the expansion coefficients. Here we study the structure and dynamics of these differential equations when the original system has Hamiltonian structure, has multiple time scales, or displays chaotic dynamics. In particular, we prove that the differential equations for the expansion coefficients in generalized polynomial chaos expansions of Hamiltonian systems retain the Hamiltonian structure relative to the ensemble average Hamiltonian. We connect this with the volume-preserving property of Hamiltonian flows to show that, for an oscillator with uncertain frequency, a finite expansion must fail at long times, regardless of the order of the expansion. Also, using a two-time scale forced nonlinear oscillator, we show that a polynomial chaos expansion of the time-averaged equations captures uncertainty in the slow evolution of the Poincaré section of the system and that, as the time scale separation increases, the computational advantage of this procedure increases. Finally, using the forced Duffing oscillator as an example, we demonstrate that when the original dynamical system displays chaotic dynamics, the resulting dynamical system from polynomial chaos also displays chaotic dynamics, limiting its applicability.