A Data-Driven Approach to Measure the Usability of Web APIs

Abstract

Application Programming Interfaces (APIs) are means of communication between applications, hence they can be seen as user interfaces, just with different kind of users, i.e., software or computers. However, the very first consumers of the APIs are humans, namely programmers. Based on the available documentation and the "ease of use" perception (sometimes led by corporate decisions and/or restrictions) they decide to use or not a specific API. In this paper, we propose a data-driven approach to measure web API usability, expressed through the predicted error rate. Following the reviewed state of the art in API usability, we identify a set of usability attributes, and for each of them we propose indicators that web API providers should refer to when developing usable web APIs. Our focus in this paper is on those indicators that can be quantified using the API logs, which indeed reflect the actual behaviour of programmers. Next, we define metrics for the aforementioned indicators, and exemplify them in our use case, applying them on the logs from the web API of District Health Information System (DHIS2) used at World Health Organization (WHO). Using these metrics as features, we build a classifier model to predict the error rate of API endpoints. Besides finding usability issues, we also drill down into the usage logs and investigate the potential causes of these errors.Index Terms-API usability, API logs, log mining, web API.

Improving Web API Usage Logging

“…Few works build their analysis on WAPI usage logs comprised by the URL requests made to the WAPI [11], [2], [19]. Thus, not only these logs' potential is still unrevealed, but even quality issues related to them or their pre-processing and preparation have not gained the deserved attention.…”

“…They reported that the results of their methods were impeded from the incomplete and noisy nature of these log data. In our previous paper [2], we proposed an approach to measure the usability of WAPIs by analyzing the usage logs generated during the development phase of consumers' application. We described the pre-processing steps of WAPI usage logs in general, and then demonstrated how we dealt with specific obstacles in the log data from the case study (e.g., data structuring, generalization).…”

“…Thus, these logs show their attempts in using the WAPI, the endpoints they struggle more with, specific mistakes they do while using and learning the WAPI, etc. [2], [19]. By analyzing these logs, providers may evaluate the usability of their WAPI from the consumers' perspective.…”

“…This paper is building upon our previous work [2], where we measured the usability of WAPIs by analyzing their usage logs generated during the development phase of consumers' applications. Based on the challenges faced while working with WAPI usage logs, and the surprising lack of attention this topic (i.e., WAPI usage logs analysis) had gained, we saw convenient to summarize our experience and research in the field, into a set of practical suggestions to enhance the logging of WAPI usage for more specialised analyses.…”

PatternLens: Inferring evolutive patterns from web API usage logs

“…But how much are these logs used and analyzed, beyond the typical traffic monitoring? Several works [8,2] suggest analyzing development logs to measure the usability of WAPIs and thus perform changes that increase the perceived usability from consumers point of view. In addition, there are various WAPI monitoring tools available that take as input the WAPI usage logs, but they are mostly oriented toward providing reporting dashboards or automatic alerting in case of WAPI failure [10].…”