We study the spectrum of the kinetic Brownian motion in the space of d × d Hermitian matrices, d ≥ 2. We show that the eigenvalues stay distinct for all times, and that the process Λ of eigenvalues is a kinetic diffusion (i.e. the pair (Λ, Λ) of Λ and its derivative is Markovian) if and only if d = 2. In the large scale and large time limit, we show that Λ converges to the usual (Markovian) Dyson Brownian motion under suitable normalisation, regardless of the dimension.