Structuring Documentation to Support State Search: A Laboratory Experiment about Protocol Programming

Abstract

Abstract. Application Programming Interfaces (APIs) often define object protocols. Objects with protocols have a finite number of states and in each state a different set of method calls is valid. Many researchers have developed protocol verification tools because protocols are notoriously difficult to follow correctly. However, recent research suggests that a major challenge for API protocol programmers is effectively searching the state space. Verification is an ineffective guide for this kind of search. In this paper we instead propose Plaiddoc, which is like Javadoc except it organizes methods by state instead of by class and it includes explicit state transitions, state-based type specifications, and rich state relationships. We compare Plaiddoc to a Javadoc control in a betweensubjects laboratory experiment. We find that Plaiddoc participants complete state search tasks in significantly less time and with significantly fewer errors than Javadoc participants.

Usability Hypotheses in the Design of Plaid

“…One benefit of static typestate information is that Javadoc-like documentation can be generated that is organized by the state the object is in, and makes state transitions explicit. In a laboratory experiment, we showed that developers were able to answer protocolrelated questions in half the time and with fewer errors when they used typestate-enhanced documentation compared to plain Javadoc [10]. Our work reinforces earlier result that also found that types provide documentation benefits [7].…”

Searching the State Space: A Qualitative Study of API Protocol Usability

“…Instead, the results of the second study are principles and understanding which we hope can be applied to any API with protocols. Our follow-on work, which validates this paper's conclusions, evaluates state-structured documentation using programming experiments that are similar to the McLellantype studies [32].…”

Interdisciplinary programming language design

“…We modified the javadoc tool to produce documentation that included an ASCII-art state machine, listed state preand post-conditions for each method, and grouped methods by state. In a controlled experiment, we found that programmers were able to answer state-related questions 2.2 times faster and were 7.9 times less likely to make errors [70]. This experiment offers the most direct evidence for the benefit of our approach, but one of the major threats to external validity is that the experiment was done in a controlled setting; how do we know that the results will transfer to the real world?…”