The surgical anatomy of the pelvis is highly complex. Anorectal and urogenital dysfunctions occur frequently after pelvic oncological surgery and are mainly caused by surgical damage of the autonomic nerves. A highly-detailed 3D pelvic model could increase the anatomical knowledge and form a solid basis for a surgical simulation system. Currently, pelvic surgeons still rely on the preoperative interpretation of 2D diagnostic images. With a 3D simulation system, pelvic surgeons could simulate and train different scenes to enhance their preoperative knowledge and improve surgical outcome. To substantially enrich pelvic surgery and anatomical education, such a system must provide insight into the relation between the autonomic network, the lymphatic system, and endopelvic fasciae. Besides CT and MR images, Visible Human Datasets (VHDs) are widely used for 3D modeling, due to the high degree of anatomical detail represented in the cryosectional images. However, key surgical structures cannot be fully identified using VHDs and radiologic imaging techniques alone. Several unsolved anatomical problems must be elucidated as well. Therefore, adequate analysis on a microscopic level is inevitable. The development of a comprehensive anatomical atlas of the pelvis is no straightforward task. Such an endeavor involves several anatomical and technical challenges. This article surveys all existing 3D pelvic models, focusing on the level of anatomical detail. The use of VHDs in the 3D reconstruction of a highly-detailed pelvic model and the accompanying anatomical challenges will be discussed.