Heuristics for evaluating IT security management tools

Jaferian, Pooya; Hawkey, Kirstie; Sotirakopoulos, Andreas; Beznosov, Konstantin

doi:10.1145/1979742.1979820

Cited by 11 publications

(24 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In 20 of the 21 cases, the authors conclude that the domain heuristics developed (or refined) are more efficient than then control heuristics. The case wherein this does not occur is in the domain of security management applications [19]; in this case only Φ P = 1.03 could be calculated, which is difficult to interpret without additional information. Table II, breaking each sub-case down as an independent input in the graph.…”

Section: A Questionnaire For Investigators That Used R3cmentioning

confidence: 99%

PROMETHEUS: Procedural Methodology For Developing Heuristics Of Usability

Jimenez

Cid

Figueroa

2017

IEEE Latin Am. Trans.

View full text Add to dashboard Cite

Usability is a key discipline related to the development of modern software systems. Its goal is to assess the userfriendliness and effectiveness of a software product from the user point of view. Therefore, proper methodologies and techniques to perform this assessment are definitely relevant. Heuristic evaluation is probably the most commonly used method for usability assessment. Initially developed by Nielsen and Molich in the '90s, traditional heuristic evaluations rely on Nielsen's wellknown 10 usability heuristics. However, recent evidence suggests that such heuristics are not sufficiently complete for dealing with new domains such as interactive television, virtual worlds, and many others. In addition to the lack of suitability of traditional heuristics, in the past years the lack of a robust methodology or process to effectively develop and validate these new domainspecific heuristics has been documented. In this paper we summarize current evidence regarding the lack of suitability of traditional heuristics, as well as the need to develop new domain-specific heuristics. After identifying and acknowledging existing gaps in heuristics for state-of-the-art technology, as documented by other researchers, we present PROMETHEUS, a PROcedural METhodology for developing HEuristics of USability. PROMETHEUS refines the methodology of , and is composed of 8 stages. PROMETHEUS clearly defines the artifacts that are required and produced by each stage, and also presents a set of quality indicators in order to assess the need for further refinement in the development of new heuristics. As an initial validation of PROMETHEUS, we apply a questionnaire to several researchers that have used the methodology of Rusu et al. , and we have also performed a small retrospective study, computing the quality indicators of several previous studies. Our results suggest that PROMETHEUS is a very promising methodology, and that the metrics and indicators are indeed pertinent with respect to the conclusions of previous studies.

show abstract

Section: A Questionnaire For Investigators That Used R3cmentioning

confidence: 99%

PROMETHEUS: Procedural Methodology For Developing Heuristics Of Usability

Jimenez

Cid

Figueroa

2017

IEEE Latin Am. Trans.

View full text Add to dashboard Cite

show abstract

“…Jaferian et al [67] point out that more complex systems may require a larger base of evaluators. In their heuristic evaluation of security management tools, between eight and ten evaluators found 75% of the usability problems, collectively.…”

Section: Heuristic Evaluationmentioning

confidence: 99%

“…Power [113] used such an approach to categorize more than 900 usability problems obtained from a heuristic evaluation. Similarly, Jaferian et al [67] categorize 164 guidelines for building IT security management tools. Preston et al [118] specifically employ thematic analysis for their pedestrian navigation data analysis.…”

Section: Thematic Analysismentioning

confidence: 99%

“…This approach varies from proposal to proposal in the thoroughness of the analysis. For instance, Jaferian et al [68] apply the two sets of heuristics to an identity management system. Other authors use more ad hoc methods.…”

Section: Assessing the MC Heuristicsmentioning

confidence: 99%

“…For our analysis, we define overlapping problems as those identified by two or more evaluators. A false alarm was identified as a problem that "was not verified by any of the researchers" [66], or when "the reasoning of the evaluator in describing the problem was fallacious" [68]. The result is a list of unique problems found for each scheme.…”

Section: )mentioning

confidence: 99%

See 2 more Smart Citations

The Usability of Captchas on Mobile Devices

Reynaga¹

View full text Add to dashboard Cite

Completely Automated Public Turing tests to tell Computers and Humans Apart (captcha) are challenge-response tests used on the web to distinguish human users from automated bots. Mobile devices such as smartphones and tablets have become a primary means of accessing online resources for many users, however most existing captchas do not properly fit mobile devices and may lead users to abandon tasks.Captchas have become sufficiently hard for users to solve that some web sites refrain from deploying them and others are actively looking at alternatives. For users of smartphones, the reduced screen size can lead to typing mistakes and loss of position. In addition, environmental context and device orientation also have an impact on the user experience.In this thesis, our research revolves around three primary, inter-related questions:How can we effectively assess usability issues of captchas accessed on smartphones?What are the most prevalent usability issues of captchas accessed on smartphones?How can we improve captchas for smartphone usage? We conducted lab and heuristic evaluations on existing and prototype captcha schemes, and identified areas for improvement. We developed, refined and tested a set of domain specific heuristics to evaluate captcha schemes on smartphones. We designed and tested four captcha prototypes to assess the viability of different input methods. From the empirical work, we identified design strategies for the development of new captcha schemes for smartphones.ii 2. We designed and implemented four prototypes of novel captcha input mechanisms [121] 3. We extracted and generalized the main design characteristics of our prototype captcha proposals that led to design strategies and recommendations for adapting captcha schemes more suitable for smartphone usage [122].

show abstract

Information Systems Architecture and Technology Security Aspects Relating to the Usability Attributes and Evaluation Methods of Mobile Commerce Websites

Goosen

Ajibola

2019

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Heuristics for evaluating IT security management tools

Cited by 11 publications

References 6 publications

PROMETHEUS: Procedural Methodology For Developing Heuristics Of Usability

PROMETHEUS: Procedural Methodology For Developing Heuristics Of Usability

The Usability of Captchas on Mobile Devices

Information Systems Architecture and Technology Security Aspects Relating to the Usability Attributes and Evaluation Methods of Mobile Commerce Websites

Contact Info

Product

Resources

About