The effect of dyslexia on information retrieval

Abstract

This is the unspecified version of the paper.This version of the publication may differ from the final published version. City University London, Northampton Square, London EC1V 0HB Permanent repository linkAbstract Purpose -The purpose of the paper is to resolve a gap in our knowledge of how people with dyslexia interact with Information Retrieval (IR) systems, specifically an understanding of their information searching behaviour. Very little research has been undertaken with this particular user group, and given the size of the group (an estimated 10% of the population) this lack of knowledge needs to be addressed. Design/Methodology/Approach -We use elements of the dyslexia cognitive profile to design a logging system recording the difference between two sets of participants: dyslexic and control users. We use a standard Okapi interface together with two standard TREC topics in order to record the information searching behaviour of these users. We gather evidence from various sources, including quantitative information on search logs, together with qualitative information from interviews and questionnaires. We record variables on queries, documents, relevance assessments and sessions in the search logs. We use this evidence to examine the difference in searching between the two sets of users, in order to understand the effect of dyslexia on the information searching behaviour. A topic analysis is also conducted on the quantitative data to show any effect on the results from the information need. Research limitations/implications -As this is a pilot study, only 10 participants were recruited for the study, 5 for each user group. Due to ethical issues, the number of topics per search was restricted to one topic only. The study shows that the methodology applied is useful for distinguishing between the two user groups, taking into account differences between topic. We outline further research on the back of this pilot study in four main areas. A different approach from the proposed methodology is needed to measure the effect on query variables, which takes account of topic variation. More details on users are needed such as reading abilities, speed of language processing and working memory to distinguish the user groups. Effect of topic on search interaction must be measured in order to record the potential impact on the dyslexic user group. Work is needed on relevance assessment and effect on precision and recall for users who may not read many documents. Findings -Using the log data, we establish the differences in information searching behaviour of control and dyslexic users i.e. in the way the two groups interact with Okapi, and that qualitative information collected (such as experience etc) may not be able to account for these differences. Evidence from query variables was unable to distinguish between groups, but differences on topic for the same variables were recorded. Users who view more documents tended to judge more documents as being relevant, either in terms of the user group or topic. Session data...

The effect of cognitive abilities on information search for tasks of varying levels of complexity

“…However, less is known about how cognitive abilities impact search. One interesting exception is the work of MacFarlane, et al [36] who, while not studying cognitive ability, studied the effects of cognitive impairment on search. They developed a logging tool based on a dyslexia-related cognitive profile and were able to distinguish the differences between the dyslexic and nondyslexic searchers.…”

“…Building upon their previous work in [36], MacFarlane, et al [37] further examined the search behaviors of dyslexic and non-dyslexic students. On average, non-dyslexic users judged more documents irrelevant than the dyslexic users (41.1 for nondyslexic versus 27.1 for dyslexic).…”

Is visual content in textual search interfaces beneficial to dyslexic users?

“…Dyslexia is usually discussed in educational contexts and in terms of teaching children how to read and write. However, dyslexics are also suggested to struggle with other activities, such as information retrieval (MacFarlane et al, 2010;Habib et al, 2012). Information search requires spelling skills in order to produce accurate and purposeful queries, and word recognition skills for exploring results and assessing documents for relevance.…”

Inferring user knowledge level from eye movement patterns

“…In the larger project of making personalized systems it is clear significant attention will need to be given to accommodation of individuals with such disorders. There is already work on the effects of dyslexia on information search (MacFarlane et al, 2010). While this is a clear limitation in our study, it could be a significant attraction for further development of cognitiveeffort eye movement-based personalization because detection of such users may be more difficult with less direct behavior observations.…”