Extremely metal-poor (EMP) stars in the Galactic halo are unique probes into the early universe and the first stars. We construct a new program to calculate the formation history of EMP stars in the early universe with the chemical evolution, based on the merging history of the Galaxy. We show that the hierarchical structure formation model reproduces the observed metallicity distribution function and also the total number of observed EMP stars, when we take into account the high-mass initial mass function and the contribution of binaries, as proposed by Komiya et al. The low-mass survivors divide into two groups of those born before and after the mini-halos are polluted by their own first supernovae. The former has observational counterparts in the hyper metal-poor (HMP) stars below [Fe/H] < −4, while the latter represents the majority of EMP stars with [Fe/H] > −4. In this Letter, we focus on the origin of the extremely small iron abundances of HMP stars. We compute the change in the surface abundances of individual stars through the accretion of the metal-enriched interstellar gas along with the dynamical and chemical evolution of the Galaxy, to demonstrate that after-birth pollution of Population III stars is sufficiently effective to explain the observed abundances of HMP stars. Metal pre-enrichment by possible pair instability supernovae is also discussed, to derive constraints on their roles and on the formation of the first low-mass stars.