Of the 6 possible orderings of the 3 main constituents of language (subject, verb and object), two —- SOV and SVO —- are predominant cross-linguistically. Previous research using the silent gesture paradigm in which hearing participants produce or respond to gestures without speech, has shown that different factors such as reversibility, salience and animacy can affect the preferences for different orders. Here, we test whether participants’ preferences for orders that are conditioned on the semantics of the event change depending on i) the iconicity of individual gestural elements and ii) the prior knowledge of a conventional lexicon. Our findings demonstrate the same preference for semantically-conditioned word order found in previous studies, specifically that SOV and SVO are preferred differentially for different types of events. We do not find that iconicity of individual gestures affects participants’ ordering preferences, however we do find that learning a lexicon leads to a stronger preference for SVO-like orders overall. Finally, we compare our findings from English speakers, using an SVO-dominant language, with data from speakers of an SOV-dominant language, Turkish. We find that, while learning a lexicon leads to an increase in SVO preference for both sets of participants, this effect is mediated by language background and event type, suggesting that an interplay of factors together determine preferences for different ordering patterns. Taken together, our results support a view of word order as a gradient phenomenon responding to multiple biases.