Purpose This study aims to develop and test a browser extension as a scaffolding tool to guide learners about evaluating online sources. Design/methodology/approach In total, 129 undergraduate students participated in this experimental study. Both groups completed two Web searching tasks, but the experimental group used a browser extension. Findings The results indicated that there are significant differences between groups in terms of the number of accurate sources and visited sites. There were no differences neither in the success status nor the access time. The browser extension guidance affected certain search parameters, but this effect seemed to be diminished in accordance with students’ cognitive abilities as well as their digital literacy levels. Research limitations/implications The participants were from a vocational school, so any other study with different participants might reveal different findings. Practical implications The browser extension is convenient to be used with regards to interface and instructions. It can serve as a self-training tool with small changes in the code. The intervals and types of messages can be customized in line with the users’ needs. Social implications The approach used in this study can contribute to the dissemination of misleading information on the Web. People of any age can use and benefit from this approach via a simple extension. Originality/value The extension can serve as a fundamental framework for the construction of adaptive or smart extensions. As this study revealed the importance of both cognitive abilities and digital literacy levels, the extension can be enriched with the inclusion of cognitive scaffolding.