There is a need to provide a more effective user interface to facilitate non-domain experts' health information seeking in authoritative online databases such as MEDLINE. We developed a new topic cluster based information navigation system called SimMed. Instead of offering a list of documents, SimMed presents users with a list of ranked clusters. Topically similar documents are grouped together to provide users with a better overview of the search results and to support exploration of similar literature within a cluster. We conducted an empirical user study to compare SimMed to a traditional document list based search interface. A total of 42 study participants were recruited to use both interfaces for health information exploration search tasks. The results showed that SimMed is more effective in terms of users' perceived topic knowledge changes and their engagement in user-system interactions. We also developed a new metric to assess users' efforts to find relevant citations. On average, users need significantly fewer clicks to find relevant information in SimMed than in the baseline system. Comments from study participants indicated that SimMed is more helpful in finding similar citations, providing related medical terms, and presenting better organized search results, particularly when the initial search is unsatisfactory. Findings from the study shed light on future health and biomedical information retrieval system and interface designs.

No abstract

Critical literacy challenges us to question how what we read has been shaped by external context, especially when information comes from less established sources. While crosschecking multiple sources provides a foundation for critical literacy, trying to keep pace the constant deluge of new online information is a daunting proposition, especially for casual readers. To help address this challenge, we propose a new form of technological assistance which automatically discovers and displays underlying memes: ideas embodied by similar phrases which are found in multiple sources. Once detected, these underlying memes are revealed to users via generated hypertext, allowing memes to be explored in context. Given the massive volume of online information today, we propose a highly-scalable system architecture based on MapReduce, extending work by Kolak and Schilit [11]. To validate our approach, we report on using our system to process and browse a 1.5 TB collection of crawled social media. Our contributions include a novel technological approach to support critical literacy and a highly-scalable system architecture for meme discovery optimized for Hadoop [25]. Our source code and Meme Browser are both available online.

Crowdsourcing has been recognized as a possible technique to complement costly user studies, usability studies, relevance judgment for information retrieval studies, and training set build-up for automatic document classification. However, the quality of crowdworkers varies by diverse factors and we often cannot tell whether their answers are right or wrong immediately due to the lack of gold standard answers. In this paper, we present a machine-learning based crowdworker filtering technique that can be used to assess workers immediately after they finish their assigned tasks. A Support Vector Machine (SVM)-based crowdworker filter, called a Smart Crowd Filter (SCFilter), was used to predict the probability that each label is correct and identifies those crowdworkers that consistently provide answers that are unlikely to be correct. To verify the performance of the SCFilter, a bad worker detection simulation test and an experiment in an actual crowdsourcing environment at the Amazon Mechanical Turk (AMT) website were performed. In the simulation test, bad worker detection performance was assessed in terms of precision and recall. In the experiment at the AMT website, a statistically significant improvement was observed for automatic document classification.

Purpose -The goals of this study are: to evaluate the merits of a newly developed health information retrieval system; to investigate users' search strategies when using the new search system; and to study the relationships between users' search strategies and their prior topic knowledge. Design/methodology/approach -The paper developed a new health information retrieval system called MeshMed. A term browser and a tree browser are included in the new system in addition to the traditional search box. The term browser allows a user to search Medical Subject Heading (MeSH) terms using natural language. The tree browser presents a hierarchical tree structure of related MeSH terms. A user study with 30 participants was conducted to evaluate the benefits of MeshMed. Findings -The paper found that MeshMed provides a user with more choices to select an appropriate searching component and form more effective search strategies. Based on the time a participant spent using different MeshMed components, the paper identified three different search styles: the traditional style, the novel style, and the balanced style, which falls in between. MeshMed was particularly helpful for users with low topic knowledge. Originality/value -A new health information retrieval system (MeshMed) was designed and developed (and is currently available at http://129.89.43.129/meshmed). This is the first study to explore users' search strategies on such a system. The study results can inform the design of future clinical-oriented health information retrieval systems.