With the widespread application of mixed-mode chromatography in separation analysis, it is becoming increasingly important to study its retention mechanism. The retention behavior of acidic compounds on mixed-mode octyl-quaternary ammonium (Sil-C8-QA) columns was investigated by computer simulation. Firstly, the benzoic acid homologues were used as the analytes, and the simulation model was constructed by the Materials Studio. Geometric optimization, annealing and molecular dynamics (MD) simulation of these complexes resulted in optimized conformations. The binding energy, mean square displacement (MSD) and torsion angle distribution generated by MD simulation were then analyzed. The results showed that the more negative binding energy, the greater the MSD and the narrower the torsion angle distribution, indicating that the stationary phase behaves with stronger interaction and retention. The retention behavior of five acidic drugs on the Sil-C8-QA column was then successfully explained by simulation. Acidic drugs are more retentive on the mixed-mode column due to the more substantial interaction brought by the reversed-phase/ion-exchange mixed-mode mechanism compared to other single-mode columns. This simulation method is expected to provide ideas for studying the separation mechanism and predicting the retention behavior of more complex samples.