The main goal of this study was to validate the strategy for the assessment of the cognitive complexity of chemical kinetics exam items. The strategy included three steps: 1) assessment of the difficulty of concepts, 2) assessment of distractor value. and 3) assessment of concepts’ interactivity. One of the tasks was to determine whether there were misconceptions by students that might have influenced their achievement. Eighty-seven students in the first year of secondary school participated in the study. A knowledge test was used as a research instrument to assess the performance, and a five-point Likert-type scale was used to evaluate the perceived mental effort. The strategy was validated using regression analysis from which significant correlation coefficients were obtained between selected variables: students’ achievement and invested mental effort (dependent variables) and a numerical rating of cognitive complexity (independent variable).