Heterocyclic organic compounds named pyrrolizidine alkaloids (PAs) belong to a group of alkaloids and are synthesized by either plants or microorganisms. Therefore, they are naturally occurring secondary metabolites. They are found in species applied in the pharmaceutical and food industries, thus a thorough knowledge of their pharmacological properties and toxicology to humans is of great importance for their further safe employment. This review is original because it synthesizes knowledge of plant and microbial PAs, which is unusual in the scientific literature. We have focused on the Boraginaceae family, which is unique due to the exceptional richness and diversity of its PAs in plant species. We have also presented the microbial sources of PAs, both from fungi and bacteria. The structure and metabolism of PAs have been discussed. Our main aim was to summarize the effects of PAs on humans, including both negative, toxic ones, mainly concerning hepatotoxicity and carcinogenicity, as well as potentially positive ones for pharmacological and medical applications. We have collected the results of studies on the anticancer activity of PAs from plant and microbial sources (mainly Streptomyces strains) and on the antimicrobial activity of PAs on different strains of microorganisms (bacteria and fungi). Finally, we have suggested potential applications and future perspectives.