Use of recycled rubber in asphalt pavement has been widely accepted since it overcame the scrap tires disposal problem and it has many advantages—for example, savings in quantity and cost of construction materials. In this study, hot-mix asphalt (HMA) specimens with or without crumb rubber modifier (CRM) were analyzed under different environmental conditions. The CRM was added into HMA by two processes (wet and dry processes). The CRM was procured from two different sources to prepare specimens (one source for each process). Cylindrical specimens were tested in a particle entrainment system at two different temperatures and three different pH levels. Slab specimens were simulated for rainfall conditions. Water samples were collected in all three phases of the study: preliminary investigations on individual CRM, water quality evaluation for cylindrical HMA specimens at various environmental conditions, and water quality evaluation for slab HMA specimens under simulated rainfall conditions. Collected water samples were analyzed for six metals: chromium, lead, nickel, copper, cadmium, and zinc. A general trend for these metals indicates that zinc was leached in higher concentrations than other metals and all the metals were leached in higher concentrations at higher temperature (maximum asphalt pavement temperature) and at lower pH (pH 2.0 conditions). Finally, based on the limited scope of this effort and comparison with water quality criteria, there does not appear to be any evidence that there will be a detrimental effect on the environment or to human health.