For an arbitrary ideal I in a polynomial ring R we define the notion of initially regular sequences on R/I. These sequences share properties with regular sequences. In particular, the length of an initially regular sequence provides a lower bound for the depth of R/I. Using combinatorial information from the initial ideal of I we construct sequences of linear polynomials that form initially regular sequences on R/I. We identify situations where initially regular sequences are also regular sequences, and we show that our results can be combined with polarization to improve known depth bounds for general monomial ideals.2010 Mathematics Subject Classification. 13C15, 13D05, 05E40, 13F20, 13P10.