We study the formation of nanostructures with alternating stripes composed of bulk-immiscible adsorbates during submonolayer heteroepitaxy. We evaluate the influence of two mechanisms considered in the literature: (i) strain relaxation by alternating arrangement of the adsorbate species, and (ii) kinetic segregation due to chemically induced diffusion barriers. A model ternary system of two adsorbates with opposite misfit relative to the substrate, and symmetric binding is investigated by off-lattice as well as lattice kinetic Monte Carlo simulations. We find that neither of the mechanisms (i) or (ii) alone can account for known experimental observations. Rather, a combination of both is needed. We present an off-lattice model which allows for a qualitative reproduction of stripe patterns as well as island ramification in agreement with recent experimental observations for CoAg/Ru(0001) [R. Q. Hwang, Phys. Rev. Lett. 76, 4757 (1996)]. The quantitative dependencies of stripe width and degree of island ramification on the misfit and interaction strength between the two adsorbate types are presented. Attempts to capture essential features in a simplified lattice gas model show that a detailed incorporation of non-local effects is required.