The oscillatory pitch motion at the leading edge of a millimeter‐scale flexible sheet on the water surface can generate undulatory locomotion for swimming, similar to a honeybee vibrating its wings for propulsion. The influence of various parameters on such swimming strategy remains unexplored. This study uses magnetic milliswimmers to probe the propulsion mechanics and impact of different parameters. It is found that this undulatory propulsion is driven by capillary forces and added mass effects related to undulatory waves of the milliswimmers, along with radiation stress stemming from capillary waves at the interface. Modifying the parameters such as actuation frequency, pitch amplitude, bending stiffness, and hydrofoil length alters the body waveform, thus, affecting the propulsion speed and energy efficiency. Although undulatory motion is not a prerequisite for water surface propulsion, optimizing body stiffness to achieve a proper undulatory waveform is crucial for efficient swimming, balancing energy consumption, and speed. The study also reveals that the induced water flow is confined near the water surface, and the flow structures evolve with varying factors. These discoveries advance the understanding of undulatory water surface propulsion and have implications for the optimal design of small‐scale swimming soft robots in the future.