This paper focuses on the further development of a previously published semi-empirical method for time domain simulation of vortex-induced vibrations (VIV). A new hydrodynamic damping formulation is given, and the necessary coefficients are found from experimental data. It is shown that the new model predicts the observed hydrodynamic damping in still water and for cross-flow oscillations in stationary incoming flow with high accuracy. Next, the excitation force model is optimized by simulating the VIV response of an elastic cylinder in a series of experiments with stationary flow. The optimization is performed by repeating the simulations until the best possible agreement with the experiments is found. The optimized model is then applied to simulate the cross-flow VIV of an elastic cylinder in oscillating flow, without introducing any changes to the hydrodynamic force modeling. By comparison with experiment, it is shown that the model predicts the frequency content, mode and amplitude of vibration with a high level of realism, and the amplitude modulations occurring at high Keulegan-Carpenter numbers are well captured. The model is also utilized to investigate the effect of increasing the maximum reduced velocity and the mass ratio of the elastic cylinder in oscillating flow. Simulations show that complex response patterns with multiple modes and frequencies appear when the maximum reduced velocity is increased. If, however, the mass ratio is increased by a factor of 5, a single mode dominates. This illustrates that, in oscillating flows, the mass ratio is important in determining the mode participation at high maximum reduced velocities.