The roughness of a flaw’s surface significantly affects the scattering behavior of ultrasonic waves. It is vital to understand the impact of roughness on flaw echoes, especially when performing ultrasonic nondestructive inspection on safety-critical components. However, the current approach for creating rough flaw models fails to reconstruct complicated cracks with secondary cracks. Here, a multi-scale distortion method is developed to generate a rough flaw by using an optical microscope image of a real flaw. The finite element (FE) is then implemented to simulate the near-surface rough flaws in nickel-based bars, which are detected by an offsetting immersion transducer with mode-converted transverse waves. Numerical results show that the randomness and complexity of flaw echoes from rough flaws are exceptionally high. The gap between the maximum and minimum normalized amplitude values of flaw echoes from a rough crack with secondary cracks can reach 7.125 dB. Meanwhile, the maximum time of flight (TOF) is almost twice as large as the minimum TOF. Therefore, the present method can generate effective rough flaw models in terms of macroscopic rough geometry and microscopic rough surface. Moreover, the impact of the rough flaw surface on the flaw echoes goes beyond amplitude changes and may make flaw location challenging.