Parameterized complexity seeks to optimally use input structure to obtain faster algorithms for NP-hard problems. This has been most successful for graphs of low treewidth, i.e., graphs decomposable by small separators: Many problems admit fast algorithms relative to treewidth and many of them are optimal under the Strong Exponential-Time Hypothesis (SETH). Fewer such results are known for more general structure such as low clique-width (decomposition by possibly large and dense but structured separators) and more restrictive structure such as low deletion distance to some sparse graph class.Despite these successes, such results remain "islands" within the realm of possible structure. Rather than adding more islands, we seek to determine the transitions between them, that is, we aim for structural thresholds where the complexity increases as input structure becomes more general. Going from deletion distance to treewidth, is a single deletion set to a graph with simple components enough to yield the same lower bound as for treewidth or does it take many disjoint separators? Going from treewidth to clique-width, how much more density entails the same complexity as clique-width? Conversely, what is the most restrictive structure that yields the same lower bound?For treewidth, we obtain both refined and new lower bounds that apply already to graphs with a single separator X such that G− X has treewidth at most r = O(1), while G has treewidth |X| + O(1). We rule out algorithms running in time O * ((r + 1 − ε) k ) for Deletion to r-Colorable parameterized by k = |X|; this implies the same lower bound relative to treedepth and (hence) also to treewidth. It specializes to O * ((3 − ε) k ) for Odd Cycle Transversal where tw(G − X) ≤ r = 2 is best possible. For cliquewidth, an extended version of the above reduction rules out time O * ((4 − ε) k ), where X is allowed to be a possibly large separator consisting of k (true) twinclasses, while the treewidth of G − X remains r; this is proved also for the more general Deletion to r-Colorable and it implies the same lower bound relative to clique-width. Further results complement what is known for Vertex Cover, Dominating Set and Maximum Cut. All lower bounds are matched by existing and newly designed algorithms.