Significant advancements have been made in the field of humanoid robot, particularly in walking control strategies. However, achieving straight‐legged walking remains a challenge. Both the traditional model‐based and the learning‐based control methods confront with difficulties in achieving natural humanoid gait feature. To address this issue, a general motion retargeting method is developed and also evaluated for humanoid robots with different structure, size and degrees of freedom. Moreover, a conditional adversarial motion priors method is proposed based on reinforcement learning and validated on the humanoid robot GTX‐III. Through various motion segments from the motion capture database, it is shown that this method can successfully enable the humanoid robot to perform straight‐legged walking with flexible and natural transitions between different gaits within a single discriminator network.