Chronic obstructive pulmonary disease (COPD) is characterized by persistent airflow obstruction and manifested by symptoms such as chronic cough, sputum, and dyspnea. Traditional Chinese medicine (TCM) has long been used to treat COPD. This article summarizes potential Chinese herbal medicines (CHM) for COPD treatment through a literature review and collation. The findings highlight 14 kinds of effective CHM for COPD, including Scutellaria radix, Salvia miltiorrhiza radix et rhizoma, Hippophae fructus, Ginseng radix et rhizoma, and Astragali radix. Flavonoids, terpenoids, alkaloids, and volatile oil constituents are the main bioactive components for the treatment of COPD. They mainly inhibit the tumor necrosis factor alpha/nuclear factor kappa B (TNF-α/NF-κB) signaling pathway by reducing the production and release of inflammatory factors such as interleukin 1 beta (IL-1β), interleukin 6 (IL-6), interleukin 8 (IL-8), TNF-α, and NF-κB, thereby reducing inflammation in lungs of rats with COPD and protect the lung tissues. Some active ingredients can inhibit toll-like receptor 4 (TLR4), mitogen-activated protein kinase (MAPK), phosphoinositide 3-kinase-protein kinase B (PI3K-Akt), Janus kinase-signal transducer and activator of transcription (JAK-STAT), and transforming growth factor beta (TGF-β) signaling pathways to express anti-inflammatory activity. This work establishes a foundation for using CHM in COPD treatment and offers new insights and approaches for developing novel anti-COPD drugs.