The reflection and transmission characteristics of a system composed of two whispering‐gallery mode (WGM) resonators are explored. Each resonator contains a Zeeman split quantum dot and side‐couples to an optical fiber. The results indicate that the unidirectional reflectionlessness and transmissionlessness can be achieved by adjusting the coupling strength between the resonators and the optical fiber, as well as the transition rate between the counter‐propagating modes within the WGM resonators. Besides, a one‐to‐one correspondence is set up between the resonance frequencies of quantum dots' energy levels and the positions of reflectionlessness (transmissionlessness) peaks when phase shift between two WGM resonators is π. This research provides the valuable insights for the development of quantum optical devices such as isolators, circulators, and routers.