This article examines the chatter isolated islands and non-linear responses of a spindle milling system supported by ball bearings. Isolated islands of chatter vibration, which are induced by the non-linear non-smooth ball bearing Hertz contacts, can be found exist in milling processes. The existence of islands of chatter vibration phenomenon is also confirmed by the bifurcation analysis in this article. The stability trends are then examined for several up-milling and down-milling machining processes, respectively, with varying machine tool radial immersions. Results show that, with the influences of the non-linear non-smooth ball bearing Hertz contacts, the milling stability charts present very complex dynamic properties and appear more irregular unstable chatter islands than those induced by the tool helix angles. It is believed that this study complements the researches of chatter vibrations for milling systems.