This article investigates a novel estimation method to optimize the position of a relay station of the S-band telemetry system in outdoor environments. To determine an optimal relay station position, complex terrain surface and buildings near the Naro Space Center are included in the EM (Electromagnetic) simulation. The optimal relay station position is then determined using the estimation method by comparing the path losses from the rocket to the main station and some candidate positions of the relay station. To better understand the path loss, the ray analysis is also conducted, and the result demonstrates that once the rocket altitude to be covered by the relay station is determined, it is possible to find the optimum position of the relay station.