Diseases such as cancer, neurological pathologies and chronic pain represent currently unmet needs. The existing pharmacotherapeutic options available for treating these conditions are limited by lack of efficiency and/or side effects. Transient receptor potential vanilloid 1 ion channel emerged as an attractive therapeutic target for developing new analgesic, anti-cancer and antiepileptic agents. Furthermore, various natural ingredients were shown to have affinity for this receptor. The aim of this narrative review was to summarize the diverse natural scaffolds of TRPV1 modulators based on their agonistic/antagonistic properties and to analyze the structure–activity relationships between the ligands and molecular targets based on the results of the existing molecular docking, mutagenesis and in vitro studies. We present here an exhaustive collection of TRPV1 modulators grouped by relevant chemical features: vanilloids, guaiacols, phenols, alkylbenzenes, monoterpenes, sesquiterpenoids, alkaloids, etc. The information herein is useful for understanding the key structural elements mediating the interaction with TRPV1 and how their structural variation impacts the interaction between the ligand and receptor. We hope this data will contribute to the design of novel effective and safe TRPV1 modulators, to help overcome the lack of effective therapeutic agents against pathologies with high morbidity and mortality.