In animal and robot swimmers of Body and Caudal Fin (BCF) form, hydrodynamic thrust is mainly produced by their caudal fins, the stiffness of which has profound effects on both thrust and efficiency of swimming. Caudal fin stiffness also affects the motor control and resulting swimming gaits that correspond to optimal swimming performance; however, their relationship remains scarcely explored. Here using magnetic, modular, undulatory robots (μBots), we tested the effects of caudal fin stiffness on both forward swimming and turning maneuver. We developed six caudal fins with stiffness of more than 3 orders of difference. For a μBot equipped with each caudal fin (and μBot absent of caudal fin), we applied reinforcement learning (RL) in experiments to optimize the motor control for maximizing forward swimming speed or final heading change. The motor control of μBot was generated by a Central Pattern Generator (CPG) for forward swimming or by a series of parameterized square waves for turning maneuver. In forward swimming, the variations in caudal fin stiffness gave rise to three modes of optimized motor frequencies and swimming gaits including no caudal fin (4.6 Hz), stiffness<10-4 Pa·m4 (~10.6 Hz) and stiffness>10-4 Pa·m4 (~8.4 Hz). Swimming speed, however, varied independently with the modes of swimming gaits, and reached maximal at stiffness of 0.23×10-4 Pa·m4, with the μBot without caudal fin achieving the lowest speed. In turning maneuver, caudal fin stiffness had considerable effects on the amplitudes of both initial head steering and subsequent recoil, as well as the final heading change. It had relatively minor effect on the turning motor program except for the μBots without caudal fin. Optimized forward swimming and turning maneuver shared an identical caudal fin stiffness and similar patterns of peduncle and caudal fin motion, suggesting simplicity in the form and function relationship in μBot swimming.