The challenge of raising a creative question exists in recombining different categories of knowledge. However, the impact of recombination remains controversial. Drawing on the theories of knowledge recombination and category-spanning, we claim that the impact of knowledge spanning on the appeal of questions is contingent upon questions’ knowledge hierarchy in the knowledge space. Using word embedding models and network analysis to quantify knowledge spanning and knowledge hierarchy respectively, we test our hypotheses with the data collected from a large online knowledge market (N = 463,545). Knowledge spanning has an inverted U-shaped influence on the appeal of questions: the appeal of questions increases up to a threshold, after which point the positive effect reverses. However, with the increase in knowledge hierarchy, the inverted U-shape is weakened and disappears quickly. We fill the research gap by conceptualizing question-asking as knowledge-spanning and highlighting the theoretical underpinnings of knowledge hierarchy. The theoretical and practical implications for future research on knowledge recombination are discussed.