L-tryptophan metabolism is involved in the regulation of many important physiological processes, such as, immune response, inflammation, and neuronal function. Indoleamine 2, 3-dioxygenase 1 (IDO1) is a key enzyme that catalyzes the first rate-limiting step of tryptophan conversion to kynurenine. Thus, inhibiting IDO1 may have therapeutic benefits for various diseases, such as, cancer, autoimmune disease, and depression. In the search for potent IDO1 inhibitors, natural quinones were the first reported IDO1 inhibitors with potent inhibitory activity. Subsequently, natural compounds with diverse structures have been found to have anti-IDO1 inhibitory activity. In this review, we provide a summary of these natural IDO1 inhibitors, which are classified as quinones, polyphenols, alkaloids and others. The overview of in vitro IDO1 inhibitory activity of natural compounds will help medicinal chemists to understand the mode of action and medical benefits of them. The scaffolds of these natural compounds can also be used for further optimization of potent IDO1 inhibitors.