This paper presents an experimental study of users assessing the quality of Google web search results. In particular we look at how users' satisfaction correlates with the effectiveness of Google as quantified by IR measures such as precision, Bpref and the suite of Cumulative Gain measures (CG, DCG, NDCG). Results indicate strong correlation between users' satisfaction, CG and precision, moderate correlation with DCG, with perhaps surprisingly negligible correlation with NDCG. The reasons for the low correlation with NDCG are examined.

The authors investigate factors influencing user satisfaction in information retrieval. It is evident from this study that user satisfaction is a subjective variable, which can be influenced by several factors such as system effectiveness, user effectiveness, user effort, and user characteristics and expectations. Therefore, information retrieval evaluators should consider all these factors in obtaining user satisfaction and in using it as a criterion of system effectiveness. Previous studies have conflicting conclusions on the relationship between user satisfaction and system effectiveness; this study has substantiated these findings and supports using user satisfaction as a criterion of system effectiveness.

Due to the COVID-19 pandemic, Higher Education Institutions (HEI) replaced regular face-to-face teaching with online teaching and learning. However, the shift caused several academic and social concerns for students, such as lack of academic support, lack of adequate resources to support online teaching, lack of socialization, stress, anxiety, and lack of motivation in attending classes. This research evaluates the impact of HEIs support, faculty support, and resources available on the academic and social concerns of students in HEIs during the pandemic. 11,114 students across the HEIs in Sultanate of Oman participated in an online national survey. Regression and factor analysis were used to verify the research model developed based on the literature review. Results showed that HEI support and faculty support significantly affect university students' academic and social concerns. Furthermore, resource availability was found to affect the academic concerns of students but not their social concerns. This research recommends strategies for HEIs and faculty to promote faculty-student interaction using both synchronous and asynchronous modes to reduce student concerns and motivate them to engage in online classes.

No abstract

This paper investigates the agreement of relevance assessments between official TREC judgments and those generated from an interactive IR experiment. Results show that 63% of documents judged relevant by our users matched official TREC judgments. Several factors contributed to differences in the agreements: the number of retrieved relevant documents; the number of relevant documents judged; system effectiveness per topic and the ranking of relevant documents.