In this contribution, the authors present the design and fabrication of optimized phase-shifting transmission masks for high-resolution nanopatterning with a compact EUV exposure tool. Several influencing factors on the achievable resolution are determined and characterized, paving the way towards the theoretical resolution limit in the sub-10 nm range. Applications that require high resolution patterns are numerous, leading to an increasing demand for compact exposure tools and lithographic concepts. The realized exposure tool is a compact and versatile setup, that can be operated either at an exposure wavelength of 10.9 nm or 13.5 nm addressing both large area nanopatterning with maximized throughput and industrial resist qualification with highest resolution. For partially coherent radiation as provided by the utilized discharge-plasma produced (DPP) EUV radiation source of the setup, the (achromatic) Talbot lithography has proven to be the most suitable lithographic approach with a demonstrated resolution in the sub-30 nm regime and a theoretical resolution limit below 10 nm. To maximize the contrast of the resulting intensity distribution in wafer plane, the material composition and geometry of the mask are optimized by means of rigorous coupled-wave (RCWA) simulations. Different influencing factors on the achievable resolution are identified and presented. In addition to the simulative optimization of the phase-shifting masks, the fabrication of the dense periodic nanopatterns becomes more and more challenging for smaller periods. In this contribution the mask fabrication process is optimized to create stable and high-resolution periodic mask patterns, leading to record resolution for both line and pinhole periodic nanopatterns with the presented setup.