The relationship between chemical structure and physical and chemical properties is essential to chemistry. Studies have shown that students have difficulty using structural representations to predict properties, which is not surprising because of the sequence of inferences that are required for sense-making. However, obtaining a nuanced model of students’ understanding depends on how information is elicited. This study investigated how the phrasing of the question prompt may elicit students’ understanding of structure–property relationships. Students were given a two-part assessment: (1) four multiple-choice questions assessing students’ self-reported abilities to predict structure–property relationships, and (2) three questions requiring students to predict, argue, and explain a boiling point trend. Two groups of students were selected to determine the sensitivity of the instrument (one with less explicit instruction of structure–property relationships and one with more explicit instruction). We found that Part I of the assessment was able to differentiate between these two groups of students. The group with more explicit instruction was further analyzed to determine how their prediction on a boiling point task connected to their arguments and explanations of the phenomenon. Even though 64% of students answered the boiling point ranking task correctly, the students typically provided less complete arguments as to why that structure had a higher boiling point. However, after scaffolding (i.e., providing relevant information for the phenomenon) and asking for an explanation, students’ responses began to include a much more mechanistic understanding, suggesting that having students provide explanations instead of constructing an argument would display their reasoning at a deeper level.