We consider a quantum spinless nonrelativistic charged particle moving in the xy plane under the action of a time-dependent magnetic field, described by means of the linear vector potential A=B(t)−y(1+α),x(1−α)/2, with two fixed values of the gauge parameter α: α=0 (the circular gauge) and α=1 (the Landau gauge). While the magnetic field is the same in all the cases, the systems with different values of the gauge parameter are not equivalent for nonstationary magnetic fields due to different structures of induced electric fields, whose lines of force are circles for α=0 and straight lines for α=1. We derive general formulas for the time-dependent mean values of the energy and magnetic moment, as well as for their variances, for an arbitrary function B(t). They are expressed in terms of solutions to the classical equation of motion ε¨+ωα2(t)ε=0, with ω1=2ω0. Explicit results are found in the cases of the sudden jump of magnetic field, the parametric resonance, the adiabatic evolution, and for several specific functions B(t), when solutions can be expressed in terms of elementary or hypergeometric functions. These examples show that the evolution of the mentioned mean values can be rather different for the two gauges, if the evolution is not adiabatic. It appears that the adiabatic approximation fails when the magnetic field goes to zero. Moreover, the sudden jump approximation can fail in this case as well. The case of a slowly varying field changing its sign seems especially interesting. In all the cases, fluctuations of the magnetic moment are very strong, frequently exceeding the square of the mean value.