Humans continuously categorise inputs, but only rarely receive explicit feedback as to whether or not they are correct. This implies that they may be integrating unsupervised information together with their sparse supervised data -- a form of semi-supervised learning. However, experiments testing semi-supervised learning are rare, and are bedevilled with conflicting results about whether the unsupervised information affords any benefit. Here, we suggest that one important factor that has been paid insufficient attention is the alignment between subjects' internal representations of the stimulus material and the experimenter-defined representations that determine success in the tasks. Subjects' representations are shaped by prior biases and experience, and unsupervised learning can only be successful if the alignment suffices. Otherwise, unsupervised learning might harmfully strengthen incorrect assumptions. To test this hypothesis, we conducted an experiment in which subjects initially categorise items along a salient, but task-irrelevant, dimension, and only recover the correct categories when sufficient feedback draws their attention to the subtle, task-relevant, stimulus dimensions. By withdrawing feedback at different stages along this learning curve, we tested whether unsupervised learning improves or worsens performance when internal stimulus representations and task are sufficiently or insufficiently aligned, respectively. Our results demonstrate that unsupervised learning can indeed have opposing effects on subjects' learning. We also discuss factors limiting the degree to which such effects can be predicted from momentary performance. Our work implies that predicting and understanding human category learning in particular tasks requires assessment and consideration of the representational spaces that subjects entertain for the materials involved in those tasks. These considerations not only apply to studies in the lab, but could also help improve the design of tutoring systems and instruction.