Exploring novel environments through sequential sampling is essential for efficient decision-making under uncertainty. In the laboratory, human exploration has been studied in situations where exploration is traded against reward maximisation. By design, these ‘explore-exploit’ dilemmas confound the behavioural characteristics of exploration with those of the trade-off itself. Here we designed a sequential sampling task where exploration can be studied and compared in the presence and absence of trade-off with exploitation. Detailed model-based analyses of choice behaviour revealed specific exploration patterns arising in situations where information seeking is not traded against reward seeking. Human choices are directed toward the most uncertain option available, but only after an initial sampling phase consisting of choice streaks from each novel option. These findings outline competing cognitive pressures on information seeking: the repeated sampling of the current option (for hypothesis testing), and the directed sampling of the most uncertain option available (for structure mapping).