The rapid increase of converter-based generation in the grid causes significant changes in the system dynamic characteristics. These changes require verification and testing of the reliability, optimization, and functionality of power system elements and controllers, which become more important for guaranteeing a secure operation. This paper first introduces a hardware-in-the-loop (HiL) setup of an automatic voltage regulator (AVR) control system and then proposes a softwarein-the-loop (SiL) setup for the parameterization of an IEEE standard AVR/excitation system model and a power oscillation damping (POD) controller of synchronous condenser. The AVR hardware is interfaced with a simulated grid in real-time digital simulator (RTDS) using EMT simulation to evaluate its control functions. Parameter optimization for the POD and the AVR simulation model by SiL simulation is implemented in a closed loop between RTDS and Matlab/Simulink interfaced through OPC to satisfy the optimization objectives. By tests executed on the future Western Danish power system, parameter optimization of the POD and IEEE standard AVR model, and the function of AVR hardware are successfully implemented and verified. Index Terms-Automatic voltage regulator, hardware-inthe-loop, parameter optimization, power oscillation damping, software-in-the-loop.