Usability Evaluation of Configuration-Based API Design Concepts

Automated measurement of API usability: The API Concepts Framework

“…They try to make use of the natural language by defining methods that are concatenated to form a readable sentence. Both of these patterns were proven to have an influence on usability [14,15,24].…”

“…The results from [15] and [14] show that when a user uses a concept for the third time, he/she has learned how to use it completely, and there is no significant further improvement after that. Therefore, a learning effect defines the complexity reduction after one use, and after two or more uses.…”

“…We therefore conducted two usability studies [15,14] to systematically evaluate some of the potential measurable properties and risks. In both studies, we tested users with both academic and industrial background, and with 2 to 20 years of programming experience.…”

“…The biggest drawback we see with this set of metrics is, that it doesn't give a comprehensive overview over the usability of an API, since most of the metrics are purely for the evaluation of methods. This may provide meaningful results for APIs that are mostly based on method calls, but we showed in [14] that there are a lot of other concepts which are used in APIs (e.g. annotations).…”

“…To identify properties that potentially influence API usability and can be measured automatically, we conduct a literature review, which is presented in section 4. For a selected subset of these properties we conducted usability studies [15,14] to show whether they really influence usability and in which way. The results of these studies are described in section 5.…”