The Virtual Population (ViP) phantoms have been used in many dosimetry studies, yet, to date, anatomical phantom uncertainty in radiofrequency (RF) research has largely been neglected. The objective of this study is to gain insight, for the first time, regarding the uncertainty in RF‐induced fields during magnetic resonance imaging associated with tissue assignment and segmentation quality and consistency in anatomical phantoms by evaluating the differences between two generations of ViP phantoms, ViP1.x and ViP3.0. The RF‐induced 10g‐average electric (E‐) fields, tangential E‐fields distribution along active implantable medical devices (AIMD) routings, and estimated AIMD heating were compared for five phantoms that are part of both ViP1.x and ViP3.0. The results demonstrated that differences exceeded 3 dB (−29%, +41%) for local quantities and 1 dB (±12% for field, ±25% for power) for integrated and volume‐averaged quantities (e.g., estimated AIMD‐heating and 10 g‐average E‐fields), while the variation across different ViP phantoms of the same generation can exceed 10 dB (−68% and +217% for field, −90% and +900% for power). In conclusion, the anatomical phantom uncertainty associated with tissue assignment and segmentation quality/consistency is larger than previously assumed, i.e., 0.6 dB or ±15% (k = 1) for AIMD heating. Further, multiple phantoms based on different volunteers covering the target population are required for quantitative analysis of dosimetric endpoints, e.g., AIMD heating, which depend on patient anatomy. Phantoms with the highest fidelity in tissue assignment and segmentation should be used, as these ensure the lowest uncertainty and possible underestimation of exposure. To verify that the uncertainty decreases monotonically with improved phantom quality, the evaluation of differences between phantom generations should be repeated for any improvement in segmentation. Bioelectromagnetics. 2019;40:458–471. © 2019 Bioelectromagnetics Society