The linear stability of a planar and a barred coast is studied with a complex processbased model. By applying either the Engelund and Hansen or the Bailard sediment transport formulation, and a wave angle of approximately 5° in the breaker zone, the sensitivity of the linear stability characteristics to the sediment transport formulation is explored. Applying Engelund and Hansen to planar beaches, no fastest growing mode is found since the growth rate continuously increases with increasing wavelength. The corresponding bed perturbations are very oblique, down-current oriented bars. With Bailard, on the contrary, the growth rate increases with decreasing wavelength, suggesting that the preferred wavelength is somewhere between 0 and 300 m. The corresponding bed forms are up-current bars. Furthermore, the results show that inclusion of sediment transport in the direction of the wave orbital motion in both formulations is crucial for the growth of bed perturbations. When only (bed load) transport in the direction of the mean current is applied, stable up-current oriented bars are found. On barred beaches, both the Engelund and Hansen, and the Bailard formulation result in growing bed perturbations, consisting of rip channels around the crest of the breaker bar. Furthermore, in both cases fastest growing modes are found. The Bailard formulation results in a smaller longshore spacing of the rip channel system than the Engelund and Hansen formulation.