A study of users’ performance and satisfaction with the Web of Science IR interface

Abstract

This paper reports on an empirical study of users' performance and satisfaction with the Web of Science interface. Two different search groups (novice and experienced) participated in the study. They carried out seven search tasks and their performance was recorded through transaction logging and computer screen recording. Data were captured on the time taken, search terms used, success score and error rates. After completion of search tasks, they completed a questionnaire on their satisfaction with the interface. The performance data showed that overall experienced users performed better than the novice group. Differences were significant in success score and error rates between the groups. Performance differences also existed on factors such as gender and previous online search training. Experienced female searchers performed best in terms of success score and error rates whereas the novice male group performed worst. Untrained users were more successful and made fewer errors than the trained group. Participants held neither highly positive nor highly negative perceptions about the Web of Science interface. Novice searchers were significantly more satisfied with the interface than the experienced group. Participants also noted both positive and negative features in the interface. This information could be used to redesign the present Web of Science interface.

A study of learning and retention with a Web-based IR interface

“…The results of the original learning phase of the study showed that novices could readily pick up the interface functionality when some training was provided. This result is consistent with earlier studies on online searching (Fenichel, 1980;Sullivan et al, 1990;Ahmed, et. al., 2004).…”

“…In contrast, Ahmed et al (2004) found that women performed better than male users. Thus, age, the level of general computer experience, sex, etc., can affect novices' search performance with IR systems.…”

“…Sullivan et al (1990) found that novices could learn to do online searches effectively after brief tutorial sessions. More recently, Ahmed et al (2004) found that novices could learn to do simple search tasks after 15-mintues of free exploration of a Web-based system. All these studies also reported novices' problems with searching online databases.…”

“…Several Web-based systems provide easy search option for novice users. Ahmed et al (2004) reported that novice users are more satisfied with the Web-based IR interfaces compared to experienced searchers. Novices' initial use of an IR system may have positively influenced their attitude with the interface.…”

Web-based interaction: A review of three important human factors

“…Regarding search perception, Ahmed, McKnight, and Oppenheim (2004) examined how users perceived the Web of Science interface, through a sample of 22 participants. The results from the questionnaire showed that the novice group was significantly more satisfied with the interface than the experienced group.…”