The magnetic moments of the octet and decuplet baryons are analyzed within the flavor SU(3) chiral quark soliton model, by putting emphasis upon the SU(3) symmetry breaking effects generated by the mass difference between the strange and nonstrange quarks. The predicted symmetry breaking patterns are shown to be quantitatively consistent with the experimental data for the octet baryons, and with the other theoretical analyses such as the chiral perturbation theory and/ or the quenched lattice simulation for the decuplet baryons.