The adsorption of single colloidal microparticles (0.5-1 µm radius) at a water-oil interface has been recently studied experimentally using digital holographic microscopy [Kaz et al., Nat. Mater., 2012, 11, 138-142]. An initially fast adsorption dynamics driven by capillary forces is followed by an unexpectedly slow relaxation to equilibrium that is logarithmic in time and can span hours or days. The slow relaxation kinetics has been attributed to the presence of surface "defects" with nanoscale dimensions (1-5 nm) that induce multiple metastable configurations of the contact line perimeter. A kinetic model considering thermally activated transitions between such metastable configurations has been proposed [Colosqui et al., Phys. Rev. Lett., 2013, 111, 028302] to predict both the relaxation rate and the crossover point to the slow logarithmic regime. However, the adsorption dynamics observed experimentally before the crossover point has remained unstudied. In this work, we propose a Langevin model that is able to describe the entire adsorption process of single colloidal particles by considering metastable states produced by surface defects and thermal motion of the particle and liquid interface. Invoking the fluctuation dissipation theorem, we introduce a drag term that considers significant dissipative forces induced by thermal fluctuations of the liquid interface. Langevin dynamics simulations based on the proposed adsorption model yield close agreement with experimental observations for different microparticles, capturing the crossover from (fast) capillary driven dynamics to (slow) thermally activated kinetics.