We have fabricated valency-controlled Si-QDs with Ge-core with an areal density as high as ~1011 cm-2 on ultrathin SiO2 and studied the effect of phosphorus- and boron-doping on Ge-core from their PL properties. During the Ge deposition, delta doping of phosphorus or boron atoms in QDs was carried out by pulse injection using 1% PH3 or B2H6 diluted with He, respectively. No changes in dot size and density with either P- or B-doping were confirmed by AFM topographic images. Under photoexcitation of undoped QDs with a 976-nm line from a semiconductor laser, broad PL spectra consisting of four Gaussian components originating from radiative recombination through quantized states in QDs were observed in the energy range from 0.62 to 0.85 eV without impurity doping even at room temperature. In the doped QDs, relatively-narrow components peaked at ~0.68 eV and ~0.64eV were observed with P-doping and with B-doping to Ge-core, respectively, in addition to the four components seen in undoped QDs. It is interesting noted that, with an increase in B2H6 pulse injection from 1 to 4 times, the integrated PL intensity was enhanced by a factor of 1.4 to 2.4 compared to that of the undoped QDs while no significant change in spectral shape was observable. This can be interpreted in terms of an increase in the number of holes with B-doping to the Ge core since the carrier recombination rates is proportional to the product of the number of electrons and holes confined in QD under weak photoexcitation.