The evolution of anisogamy (the production of gametes of different size) is the first step in the establishment of sexual dimorphism, and it is a fundamental phenomenon underlying sexual selection. It is believed that anisogamy originated from isogamy (production of gametes of equal size), which is considered by most theorists to be the ancestral condition. Although nearly all plant and animal species are anisogamous, extant species of marine green algae exhibit a diversity of mating systems including both isogamy and anisogamy. Isogamy in marine green algae is of two forms: isogamy with extremely small gametes and isogamy with larger gametes. Based on disruptive selection for fertilization success and zygote survival (theory of Parker, Baker, and Smith), we explored how environmental changes can contribute to the evolution of such complex mating systems by analyzing the stochastic process in the invasion simulations of populations of differing gamete sizes. We find that both forms of isogamy can evolve from other isogamous ancestors through anisogamy. The resulting dimensionless analysis accounts for the evolutionary stability of all types of mating systems in marine green algae, even in the same environment. These results imply that evolutionary trajectories as well as the optimality of gametes/zygotes played an important role in the evolution of gamete size. convergent evolution | evolutionarily stable strategy | fertilization kinetics