When Relevance Judgement is Happening?

Allegretti, Marco; Moshfeghi, Yashar; Hadjigeorgieva, Maria; Pollick, Frank E.; Jose, Joemon M.; Pasi, Gabriella

doi:10.1145/2766462.2767811

Cited by 30 publications

(32 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Likewise, Kauppi et al [25] showed using MEG that the frequency content of the MEG signal, along with eye movement data can be used for decoding the relevance of images. These results are consistent with the findings of Allegretti et al [1] who reported on EEG results that indicated that within 500 ms EEG signals begin to appear that differentiate between viewing a relevant and a non-relevant image. These studies with fMRI, EEG and MEG have used the relative strength of the different measurement techniques to make great progress and to indicate where in the brain relevance judgments are happening and what the time course is of these neural processes that determine relevance.…”

Section: Advances In Neuroscience and Irsupporting

confidence: 92%

Towards Predicting a Realisation of an Information Need based on Brain Signals

Moshfeghi

Triantafillou

Pollick

2019

The World Wide Web Conference

Self Cite

View full text Add to dashboard Cite

The goal of Information Retrieval (IR) systems is to satisfy searchers' Information Need (IN). Our research focuses on next-generation IR engines, which can proactively detect, identify, and serve INs without receiving explicit queries. It is essential, therefore, to be able to detect when INs occur. Previous research has established that a realisation of INs physically manifests itself with specific brain activity. With this work we take the next step, showing that monitoring brain activity can lead to accurate predictions of a realisation of IN occurrence. We have conducted experiments whereby twentyfour participants performed a Q/A Task, while their brain activity was being monitored using functional Magnetic Resonance Imaging (fMRI) technology. The questions were selected and developed from the TREC-8 and TREC 2001 Q/A Tracks. We present two methods for predicting the realisation of an IN, i.e. Generalised method (GM) and Personalised method (PM). GM is based on the collective brain activity of all twenty-four participants in a predetermined set of brain regions known to be involved in representing a realisation of INs. PM is unique to each individual and employs a 'Searchlight' analysis to locate brain regions informative for distinguishing when a "specific" user realises an information need. The results of our study show that both methods were able to predict a realisation of an IN (statistically) significantly better than chance. Our results also show that PM (statistically) significantly outperformed GM in terms of prediction accuracy. These encouraging findings make the first fundamental step towards proactive IR engines based on brain signals.

show abstract

Section: Advances In Neuroscience and Irsupporting

confidence: 92%

Towards Predicting a Realisation of an Information Need based on Brain Signals

Moshfeghi

Triantafillou

Pollick

2019

The World Wide Web Conference

Self Cite

View full text Add to dashboard Cite

show abstract

“…The differences in P600 amplitudes were significant between the two types of shots in the frontal‐central and central lobes at a Bonferroni‐corrected alpha level of 0.0017. The locations of the activated electrodes appear to reflect a fronto‐central P600 effect, similar to the report by Allegretti et al (), who suggested that a central P600 effect occurs when making relevance judgments for visual stimuli. Ruchkin, Johnson, Mahaffey, and Sutton () reported that a late frontal positivity is associated with memory retrieval tasks for graphical stimuli.…”

Section: Discussionsupporting

confidence: 87%

“…As the two early attention and perception phases seem to be automatic or near-automatic responses to stimuli, we did not expect significant differences between two different conditions in these two phases (Allegretti et al, 2015). Thus, we focused our hypotheses on the integration phase and excluded the early two phases from our analysis.…”

Section: Research Hypothesesmentioning

confidence: 99%

Video summarization using event‐related potential responses to shot boundaries in real‐time video watching

Kim

2018

Asso for Info Science & Tech

View full text Add to dashboard Cite

Our aim was to develop an event‐related potential (ERP)‐based method to construct a video skim consisting of key shots to bridge the semantic gap between the topic inferred from a whole video and that from its summary. Mayer's cognitive model was examined, wherein the topic integration process of a user evoked by a visual stimulus can be associated with long‐latency ERP components. We determined that long‐latency ERP components are suitable for measuring a user's neuronal response through a literature review. We hypothesized that N300 is specific to the categorization of all shots regardless of topic relevance, N400 is specific for the semantic mismatching process for topic‐irrelevant shots, and P600 is specific for the context updating process for topic‐relevant shots. In our experiment, the N400 component led to more negative ERP signals in response to topic‐irrelevant shots than to topic‐relevant shots and showed a fronto‐central scalp pattern. P600 elicited more positive ERP signals for topic‐relevant shots than for topic‐irrelevant shots and showed a fronto‐central scalp pattern. We used discriminant and artificial neural network (ANN) analyses to decode video shot relevance and observed that the ANN produced particularly high success rates: 91.3% from the training set and 100% from the test set.

show abstract

“…Similarly, Allegretti et al [1] reported on EEG results that indicated that within 500 ms EEG signals begin to appear that differentiate between viewing a relevant and a non relevant image. Likewise, Kauppi et al [23] used magnetoencephalography (MEG) to show that the frequency content of the MEG signal, along with eye movement data can be used for decoding relevance of images.…”

Section: Neuropsychology and Irmentioning

confidence: 94%

Understanding Information Need

Moshfeghi

Triantafillou

Pollick

2016

Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval

Self Cite

View full text Add to dashboard Cite

The raison d'etre of IR is to satisfy human information need. But, do we really understand information need? Despite advances in the past few decades in both the IR and relevant scientific communities, this question is largely unanswered. We do not really understand how an information need emerges and how it is physically manifested. Information need refers to a complex concept: at the very initial state of the phenomenon (i.e. at a visceral level), even the searcher may not be aware of its existence. This renders the measuring of this concept (using traditional behaviour studies) nearly impossible. In this paper, we investigate the connection between an information need and brain activity. Using functional Magnetic Resonance Imaging (fMRI), we measured the brain activity of twenty four participants while they performed a Question Answering (Q/A) Task, where the questions were carefully selected and developed from TREC-8 and TREC 2001 Q/A Track. The results of this experiment revealed a distributed network of brain regions commonly associated with activities related to information need and retrieval and differing brain activity in processing scenarios when participants knew the answer to a given question and when they did not and needed to search. We believe our study and conclusions constitute an important step in unravelling the nature of information need and therefore better satisfying it.

show abstract

When Relevance Judgement is Happening?

Cited by 30 publications

References 12 publications

Towards Predicting a Realisation of an Information Need based on Brain Signals

Towards Predicting a Realisation of an Information Need based on Brain Signals

Video summarization using event‐related potential responses to shot boundaries in real‐time video watching

Understanding Information Need

Contact Info

Product

Resources

About