Using time-resolved transconductance spectroscopy, we study the tunneling dynamics between a two-dimensional electron gas (2DEG) and self-assembled quantum dots (QDs), embedded in a field-effect transistor structure. We find that the tunneling of electrons from the 2DEG into the QDs is governed by a different time constant than the reverse process, i.e., tunneling from the QDs to the 2DEG. This asymmetry is a clear signature of Coulomb interaction and makes it possible to determine the degeneracy of the quantum dot orbitals even when the individual states cannot be resolved energetically because of inhomogeneous broadening. Our experimental data can be qualitatively explained within a master-equation approach.