Growing empirical evidence shows that ensemble information (e.g., the average feature or feature variance of a set of objects) affects visual working memory for individual items. Recently, Harrison, McMaster, and Bays (2021) used a change detection task to test whether observers explicitly rely on ensemble representations to improve their memory for individual objects. They found that sensitivity to simultaneous changes in all memorized items (which also entail changes in set summary statistics) rarely exceeded a level predicted by the so-called optimal summation model. This model implies simple integration of evidence for change from all individual items but without any additional evidence coming from ensemble. Here, we argue that performance at the optimal summation level does not rule out the use of ensemble information. First, in two experiments, we show that, even if evidence from only one item is available at test, the statistics of the whole memory set affect performance. Second, we argue that the optimal decision strategy described by Harrison et al. is at least partly ensemble-based, whereas a strictly item-based strategy (the so-called minimum rule) predicts much lower sensitivity that both our and Harrison et al. (2021)’s observers consistently outperformed. We conclude that observers can encode ensemble information into working memory and rely on it at test.