Populus species are well documented for being potentially suitable for phytoremediation purposes regarding their accumulation characteristics. However, published results are contradictory. Based on the data gathered during an extensive literature search, we aimed to assess and revise the metal accumulation potential in the root, stem, and leaf of Populus species growing in contaminated soils, with meta-analysis. We evaluated the influences of pollution level, soil pH, and exposure time on the metal uptake patterns. We found accumulations of Cd, Cr, Cu, Pb, and Zn to be significant in each plant part, while that was only moderate for Ni, and limited for Mn. By calculating the soil pollution index (PI), we observed significantly intensive, PI-independent accumulation for Cd, Cr, Cu, Ni, Pb, and Zn. A decrease in soil pH significantly increased the uptake of Mn and significantly decreased the accumulation of Pb in the stem. Metal uptake was significantly influenced by exposure time as well; Cd concentration was significantly decreased in the stem, while concentrations of Cr in the stem and leaf, and Mn in the stem were significantly increased with time. These aforementioned findings support a well-founded metal-and-growth condition-specific application of poplars in phytoremediation processes, also triggering further in-depth assessments to enhance the efficiency of relevant poplar-based technologies.