Modern Japanese are considered to derive from a mixture of two major ancestral populations: the indigenous Jomon people and immigrants from continental East Asia. Since most of the existing methods for detecting genetic components from ancestral populations require their genomes, ancestral genomic components in Japanese could not detected so far due to the lack of precisely sequenced ancient Jomon genomes. To overcome the difficulty, we developed a reference-free detection method using a novel summary statistic, the ancestry-marker index (AMI). We applied the AMI to modern Japanese samples from the 1000 Genomes Project and identified 208,648 ancestry-marker SNPs that were likely derived from the Jomon people (Jomon-derived SNPs). Comparing the Jomon allele score detected in this study with modern Japanese and two ancient Jomon individuals showed that the Jomon derived SNPs were detected with high accuracy by the AMI in real data, and that the Jomon derived SNPs were detected by several tens of times from a single Jomon individual by the AMI. The analysis of Jomon-derived SNPs in 10,842 modern Japanese individuals recruited from all the 47 prefectures of Japan showed that the genetic differences among the prefectures were mainly caused by differences in the admixture proportion of the Jomon people, due to the difference of population size of immigrants in the final Jomon to the Yayoi period. We also confirmed the presence of the Jomon alleles around phenotype associated SNPs characteristic of East Asians to clarify whether these phenotypes of modern Japanese were derived from the Jomon people.