Necessary and sufficient conditions are given for the uniform convergence over an arbitrary index set in von Neumann's mean and Birkhoff's pointwise ergodic theorem. Three different types of conditions already known from probability theory are investigated. Firstly it is shown that the property of being eventually totally bounded in the mean is necessary and sufficient. This condition involves as a particular case Blum-DeHardt's theorem which offers the best known sufficient condition for the uniform law of large numbers in the independent case. Secondly it is shown that eventual tightness is necessary and sufficient. In this way a link with a weak convergence is obtained. Finally it is shown that the existence of some particular totally bounded pseudo-metrics is necessary and sufficient. The conditions derived are of Lipschitz type, while the method of proof relies upon a result of independent interest, called the uniform ergodic lemma. This result considerably extends Hopf-Yosida-Kakutani's maximal ergodic lemma to a form more suitable for examinations of the uniform convergence under consideration. From this lemma an inequality is also derived which extends the classical (weak) maximal ergodic inequality to the uniform case. In addition, a uniform approximation by means of a dense family of maps satisfying the uniform ergodic theorem in a trivial way is investigated, and a particular result of this type is established. This approach is in the spirit of the classical Hilbert space method for the mean ergodic theorem of von Neumann, and therefore from the ergodic theory point of view it could be seen as the natural one. After this, a simple characterization is obtained for the uniform convergence of moving averages. Finally, a counter-example is constructed for a symmetrization inequality in the stationary ergodic case. This inequality is known to be of vital importance to support the Vapnik-Chervonenkis random entropy approach in the independent case. Further developments in this direction are indicated.