Claims Analysis "In the Wild:" A Case Study on Digital Library Development

Proceedings of the 4th ACM/IEEE-CS Joint Conference on Digital Libraries

Keith

Connell

et al. 2004

Self Cite

There are two main kinds of approach to considering usability of any system: empirical and analytical. Empirical techniques involve testing systems with users, whereas analytical techniques involve usability personnel assessing systems using established theories and methods. We report here on a set of studies in which four different techniques were applied to various digital libraries, focusing on the strengths, limitations and scope of each approach. Two of the techniques, Heuristic Evaluation and Cognitive Walkthrough, were applied in text-book fashion, because there was no obvious way to contextualize them to the Digital Libraries (DL) domain. For the third, Claims Analysis, it was possible to develop a set of re-usable scenarios and personas that relate the approach specifically to DL development. The fourth technique, CASSM, relates explicitly to the DL domain by combining empirical data with an analytical approach. We have found that Heuristic Evaluation and Cognitive Walkthrough only address superficial aspects of interface design (but are good for that), whereas Claims Analysis and CASSM can help identify deeper conceptual difficulties (but demand greater skill of the analyst). However, none fit seamlessly with existing digital library development practices, highlighting an important area for further work to support improved usability.

Section: Findings and Discussionmentioning

confidence: 77%

Analytical usability evaluation for digital libraries

Proceedings of the 4th ACM/IEEE-CS Joint Conference on Digital Libraries

Keith

Connell

et al. 2004

Self Cite

“…Similarly, Blandford et al (2008) present a worked example of how CASSM can be applied to think‐aloud data of postgraduate HCI and library and information studies students using a range of electronic resources. Blandford and her colleagues have also tailored claims analysis to a digital libraries context (see Blandford, Keith, & Fields, 2006; Blandford, Keith, Fields, & Furniss, 2007).…”

Section: The Ib Methodsmentioning

confidence: 99%

Using information behaviors to evaluate the functionality and usability of electronic resources: From Ellis's model to evaluation

Makri

Cox

2008

J. Am. Soc. Inf. Sci.

Self Cite

Information behavior (IB) research involves examining how people look for and use information, often with the sole purpose of gaining insights into the behavior displayed. However, it is also possible to examine IB with the purpose of using the insights gained to design new tools or improve the design of existing tools to support information seeking and use. This approach is advocated by David Ellis who, over two decades ago, presented a model of information seeking behaviors and made suggestions for how electronic tools might be designed to support these behaviors. Ellis also recognized that IBs might be used as the basis for evaluating as well as designing electronic resources. In this article, we present the IB evaluation methods. These two novel methods, based on an extension of Ellis's model, use the empirically observed IBs of lawyers as a framework for structuring user-centered evaluations of the functionality and usability of electronic resources. In this article, we present the IB methods and illustrate their use through the discussion of two examples. We also discuss benefits and limitations, grounded in specific features of the methods. IntroductionIn this article, we present the information behavior (IB) methods-two novel, specialized methods for evaluating electronic resources ranging from Internet search engines to digital libraries and indexes/citators. The IB methods are novel as they are based on the observed IB of lawyers and are theoretically underpinned by an extension of Ellis's behavioral model of information seeking. The methods are specialized in the sense that they are intended to be used to evaluate electronic resources as opposed to other types of interactive system. The IB methods were developed based on the premise that observed IB can provide a useful structure for evaluating electronic resources by using the observed behaviors as theoretical "lenses" to evaluate the functionality Received April 8, 2008; revised June 20, 2008; accepted June 20, 2008 © 2008 ASIS&T • Published online 7 August 2008 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/asi.20927 of electronic resources (i.e., the features provided by the resource aimed at supporting users) and the usability of these resources (i.e., how easy to use the resource is).Although both the IB functionality and usability methods are underpinned by the same IB theory, they can be regarded as two separate evaluation methods in their own right. An IB functionality evaluation aims to provide data on the range of functionality supported by a particular electronic resource (and provides the basis for discussing whether this range is appropriate). An IB usability evaluation aims to provide data relating to the difficulties that users face when using a particular resource and how severe and easy to address the evaluator considers these issues to be.We continue this article by discussing the rationale behind the IB methods and the theoretical basis of the methods. This includes reference to the empirically observed IBs o...

“…Olmsted-Hawala et al 2010); case studies of method adoption and appropriation (e.g. John and Packer 1995;Spencer 2000;Blandford, Keith, and Fields 2006;Berndt, Furniss, and Blandford 2015); studies of method use in practice (e.g. Nùrgaard and Hornbaek 2006) and new conceptions of what 'methods' actually mean (e.g.…”

Section: Understanding Human Factors Practice and Usability Evaluatiomentioning

confidence: 99%

Using FRAM beyond safety: a case study to explore how sociotechnical systems can flourish or stall

Furniss

Curzon

Theoretical Issues in Ergonomics Science

2016

Self Cite

Functional Resonance Analysis Method (FRAM) is a relatively new method that has been proposed to explore how functional variability can escalate into unexpected, and often unwanted, events. It has been used for accident analyses and risk assessments in safety. We apply (and slightly modify) FRAM, to analyse how functions are configured to create systems that excel. Our case study focuses on how functions in human factors project work positively resonate to improve the delivery of value. From interviews with 22 practitioners we derived 29 functions and 6 subsystems showing how functions are coupled. Practitioners validated this model through respondent validation. Our case study evaluates the applicability and usability of FRAM. It shows how we adapted the method to make it more usable. It shows that FRAM can be used to examine positive and negative resonances in systems, to investigate how complex sociotechnical systems can flourish or stall. KEYWORDS FRAM; resilience engineering; human factors methods, performance; quality management Relevance to human factors/ergonomics theory This paper explores a new use of Functional Resonance Analysis Method (FRAM). Whereas previous uses have focused on accident analysis and risk assessment, we use FRAM to explore successful outcomes and gains in effectiveness. We show how FRAM can be used to investigate how complex sociotechnical systems flourish or stall.CONTACT Dominic Furniss d.furniss@ucl.ac.uk