Magnetic track brakes work independently of the wheel–rail contact as an additional braking system in railway vehicles. In the past, magnetic track brakes were usually deactivated at velocities below 25 km/h in mainline applications to avoid stopping jerks. However, current demands on braking performance require activation until full stop. During field tests on this subject, severe self-excited vibrations were measured at velocities below 25 km/h. This study analyses the oscillations observed by focusing on the stability behaviour of a simplified linear coupled electro-magneto-mechanical model of the track brake. Key parameters are identified by applying established criteria of linear stability theory. To account for essential nonlinear effects, a detailed multibody dynamics model with flexible bodies including a more enhanced electro-magnetic model is introduced and parameterised by using measurement data. Simulation results reveal two separate (or combined) mechanisms that may lead to self-excitation. On the one hand, friction-induced vibrations between the magnets and the rail; on the other hand, coupling effects between the electro-magnetic and the mechanical subsystems of the track brake may be the cause of the self-excited vibrations observed. Stability behaviour is influenced by various design parameters and in particular by the contact conditions between track brake and rail. Finally, a few passive methods are briefly discussed that may help to mitigate or reduce self-excited vibrations at low velocities in future designs of track brakes.