We report a photoluminescence (PL) spectroscopy study of charge state control in single selfassembled InAs/GaAs quantum dots by applying electric and/or magnetic fields at 4.2 K. Neutral and charged exciton complexes were observed under applied bias voltages from -0.5 V to 0.5 V by controlling the carrier tunneling. The highly negatively charged exciton emission becomes stronger with increasing pumping power, arising from the fact that electrons have a smaller effective mass than holes and are more easily captured by the quantum dots. The integrated PL intensity of negatively charged excitons is affected significantly by a magnetic field applied along the sample growth axis. This observation is explained by a reduction in the electron drift velocity caused by an applied magnetic field, which increases the probability of non-resonantly excited electrons being trapped by localized potentials at the wetting layer interface, and results in fewer electrons distributed in the quantum dots. The hole drift velocity is also affected by the magnetic field, but it is much weaker.