We propose a new ES-BGK model for diatomic gases which allows for translational-rotational and translational-vibrational energy exchanges, as given by Landau-Teller and Jeans relaxation equations. This model is consistent with the general definition of the vibrational and rotational collision numbers that are also commonly used in DSMC solvers. It is proved to satisfy the H-theorem and to give the correct transport coefficients, up to the volume viscosity.