The paper presents new knowledge on primary defect formation in tungsten (W) and iron (Fe) irradiated by fission and high-energy neutrons at near-room temperature. Using a well-established method of positron-annihilation lifetime-spectroscopy (PALS), it was found that irradiation of W in the fission reactor and by high-energy neutrons from the p(35 MeV)-Be generator leads to the formation of small radiation-induced vacancy clusters with comparable mean size. In the case of Fe, smaller mean size of primary radiation-induced vacancy clusters was measured after irradiation with fission neutrons compared to irradiation with high-energy neutrons from the p(35 MeV)-Be generator. It was found that one of the reasons of the formation of the larger size of the defects with lower density in Fe is lower flux in the case of irradiation with high-energy neutrons from the p(35 MeV)-Be source. The second reason is enhanced defect agglomeration and recombination within the energetic displacement cascade at high energy primary knock-on-atoms (PKAs). This is consistent with the concept of the athermal recombination corrected (arc-dpa) model, although the measured dpa cross-section of both fission neutrons and wide-spectrum high-energy neutrons in W is between the conventional Norgett–Robinson–Torrens (NRT-dpa) and arc-dpa predictions. This means that the physics of the primary radiation effects in materials is still not fully known and requires further study through a combination of modeling and experimental efforts. The present data serve as a basis for the development of an improved concept of the displacement process.