Purpose The purpose of this paper is to provide information about the long-term ex-post impacts of Vision 2023 technology foresight (TF) on the defense sector and to identify critical success factors (CSFs) of impactful foresight. Design/methodology/approach In the present research, a theory-based evaluation approach was adopted with the logic-model of defense TF to identify the intended outcomes, impacts and leading mechanisms. The impact assessment framework developed by Johnston, R. (2012) was adopted to identify (un)intended impacts and possible measures. Findings TF had some effects on technological developments, foresight capacity and capability and skills on the sector. The overall impact was assessed at the “some contribution” level with 2.9 out of 5.0 points. It contributed to the development of science technology and innovation (STI) policies and research and development programs, awareness-raising in STI, increase in cooperation between government-university-industry and the development of foresight culture. However, the impacts were more visible in the early years of TF. Additionally, country/sector-specific CSFs were identified. In consequence, it was proposed to measure the maturity of strategic technologies with technology readiness level as a tangible indicator. Originality/value According to the authors, this is the first study to assess the long-term ex-post impact of TF in defense. An instrument was developed to assess TF’s contribution to impact measures. The constructs and weights of the instrument differentiated from the adopted framework/schema reflecting the national/sectoral context of TF. Additionally, the study revealed country/sector-specific CSFs and new tangible impact measures.