A Generic Cognitive Dimensions Questionnaire to Evaluate the Usability of Security APIs

Abstract

Usability issues that exist in security APIs cause programmers to embed those security APIs incorrectly to the applications they develop. This results in introduction of security vulnerabilities to those applications. One of the main reasons for security APIs to be not usable is currently there is no proper method by which the usability issues of security APIs can be identified. We conducted a study to assess the effectiveness of the cognitive dimensions questionnaire based usability evaluation methodology in evaluating the usability of security APIs. We used a cognitive dimensions based generic questionnaire to collect feedback from programmers who participated in the study. Results revealed interesting facts about the prevailing usability issues in four commonly used security APIs and the capability of the methodology to identify those issues.

Fighting Against XSS Attacks. A Usability Evaluation of OWASP ESAPI Output Encoding

“…Conducting a user study is a widely known method for identifying usability issues of APIs [14,15,21]. In a user study based usability evaluation, evaluators will recruit programmers and ask them to complete some tasks that will require them to use the API under evaluation.…”

“…The cognitive dimensions framework presents a set of dimensions that describe aspects of a tool or an API that impact its usability [14,31]. We used the version of cognitive dimensions framework proposed by Wijayarathna, Arachchilage and Slay [14] in this study, which consists of 15 cognitive dimensions. This framework is embedded to the usability evaluation process through the cognitive dimensions questionnaire [14,31].…”

“…The relationship between the usability of security APIs (APIs that provide security related functionalities) and security of end user applications that use those security APIs has become a topic of high interest among researchers recently [14,15,16,17,18,19]. There have been several studies that discuss and investigate this relationship [9,15,16,19,20].…”

A methodology to Evaluate the Usability of Security APIs

“…Therefore, we recommend to use a predefined generic questionnaire based on the CDF to collect feedback about the usability of the evaluated API from participants who complete the task/tasks using the API. However, the questionnaire used by Clarke [11] can not be used to evaluate the usability of security APIs, because there are more usability aspects that needs to be considered when evaluating security APIs that are not included in the Clarke's framework and questionnaire [3]- [5]. Wijayarathna et al [3] proposed an enhanced version of the Clarke's framework and questionnaire by including security API related usability aspects proposed by Green and Smith [5], and Gorski and Iacono [4] to use in the usability evaluations of security APIs.…”

“…Since most of the programmers who are involved in the software development process are not experts of cyber security and related aspects [2], they use Application Programming Interfaces (APIs) that provide security related functionalities to embed security functionalities to applications they develop [2]. These APIs that provide security related functionalities are known as security APIs [3], [4]. When security APIs that programmers use are not usable, it is difficult for programmers to learn and use APIs and hence, leads them to make mistakes that would result in introducing security vulnerabilities to applications they develop [4], [7], [30].…”