Asthma control is suboptimal among World Trade Center Health Registry (WTCHR) enrollees. Air pollution/irritants have been reported as the most prevalent trigger among World Trade Center responders. We examined the relationship between air pollution/irritants and asthma control. We also evaluated the association of asthma control with health-related quality of life (HRQoL). We included 6202 enrollees age ≥18 with a history of asthma who completed the WTCHR asthma survey between 2015 and 2016. Based on modified National Asthma Education and Prevention Program criteria, asthma was categorized as controlled, poorly-controlled, or very poorly-controlled. HRQoL indicators include ≥14 unhealthy days, ≥14 activity limitation days, and self-rated general health. We used multinomial logistic regression for asthma control, and unconditional logistic regression for HRQoL, adjusting for covariates. Overall, 27.1% had poorly-controlled and 32.2% had very poorly-controlled asthma. Air pollution/irritants were associated with poorly-controlled (adjusted odds ratio (AOR) = 1.70; 95% CI = 1.45–1.99) and very poorly-controlled asthma (AOR = 2.15; 95% CI = 1.83–2.53). Poor asthma control in turn worsened the HRQoL of asthmatic patients. Very poorly-controlled asthma was significantly associated with ≥14 unhealthy days (AOR = 3.60; 95% CI = 3.02–4.30), ≥14 activity limitation days (AOR = 4.37; 95% CI = 3.48–5.50), and poor/fair general health status (AOR = 4.92; 95% CI = 4.11–5.89). Minimizing World Trade Center (WTC) asthmatic patients’ exposure to air pollution/irritants may improve their disease management and overall well-being.