Experiments play a central role in science. The role of experiments
in computing is, however, unclear. Questions about the relevance of experiments in computing attracted little attention until the 1980s. As the
discipline then saw a push towards experimental computer science, a variety of technically, theoretically, and empirically oriented views on experiments emerged. As a consequence of those debates, today's computing
fields use experiments and experiment terminology in a variety of ways.
This paper analyzes experimentation debates in computing. It presents
five ways in which debaters have conceptualized experiments in computing: feasibility experiment, trial experiment, field experiment, comparison
experiment, and controlled experiment. This paper has three aims: to
clarify experiment terminology in computing; to contribute to disciplinary
self-understanding of computing; and, due to computing's centrality in
other fields, to promote understanding of experiments in modern science
in general.