Radiation dose calculation using models of the human anatomy has been a subject of great interest to radiation protection, medical imaging, and radiotherapy. However, early pioneers of this field did not foresee the exponential growth of research activity as observed today. This review article walks the reader through the history of the research and development in this field of study which started some 50 years ago. This review identifies a clear progression of computational phantom complexity which can be denoted by three distinct generations. The first generation of stylized phantoms, representing a grouping of less than dozen models, was initially developed in the 1960s at Oak Ridge National Laboratory to calculate internal doses from nuclear medicine procedures. Despite their anatomical simplicity, these computational phantoms were the best tools available at the time for internal/external dosimetry, image evaluation, and treatment dose evaluations. A second generation of a large number of voxelized phantoms arose rapidly in the late 1980s as a result of the increased availability of tomographic medical imaging and computers. Surprisingly, the last decade saw the emergence of the third generation of phantoms which are based on advanced geometries called boundary representation (BREP) in the form of Non-Uniform Rational B-Splines (NURBS) or polygonal meshes. This new class of phantoms now consists of over 287 models including those used for non-ionizing radiation applications. This review article aims to provide the reader with a general understanding of how the field of computational phantoms came about and the technical challenges it faced at different times. This goal is achieved by defining basic geometry modeling techniques and by analyzing selected phantoms in terms of geometrical features and dosimetric problems to be solved. The rich historical information is summarized in four tables that are aided by highlights in the text on how some of the most well-known phantoms were developed and used in practice. Some of the information covered in this review has not been previously reported, for example, the CAM and CAF phantoms developed in 1970s for space radiation applications. The author also clarifies confusion about “population-average” prospective dosimetry needed for radiological protection under the current ICRP radiation protection system and “individualized” retrospective dosimetry often performed for medical physics studies. To illustrate the impact of computational phantoms, a section of this article is devoted to examples from the author’s own research group. Finally the author explains an unexpected finding during the course of preparing for this article that the phantoms from the past 50 years followed a pattern of exponential growth. The review ends on a brief discussion of future research needs (A supplementary file “3DPhantoms.pdf” to Figure 15 is available for download that will allow a reader to interactively visualize the phantoms in 3D).