Duplicate Question Detection in Stack Overflow: A Reproducibility Study

Abstract

Stack Overflow has become a fundamental element of developer toolset. Such influence increase has been accompanied by an effort from Stack Overflow community to keep the quality of its content. One of the problems which jeopardizes that quality is the continuous growth of duplicated questions. To solve this problem, prior works focused on automatically detecting duplicated questions. Two important solutions are DupPredictor and Dupe. Despite reporting significant results, both works do not provide their implementations publicly available, hindering subsequent works in scientific literature which rely on them. We executed an empirical study as a reproduction of DupPredictor and Dupe. Our results, not robust when attempted with different set of tools and data sets, show that the barriers to reproduce these approaches are high. Furthermore, when applied to more recent data, we observe a performance decay of our both reproductions in terms of recall-rate over time, as the number of questions increases. Our findings suggest that the subsequent works concerning detection of duplicated questions in Question and Answer communities require more investigation to assert their findings.

Recommending Comprehensive Solutions for Programming Tasks by Mining Crowd Knowledge

“…k and b are two parameters where k controls non-linear term frequency normalization (saturation), and b controls to what degree document length normalizes term frequency values. CROKAGE uses the same values as used by previous works in Software Engineering [13,14] with best performance. idf (q i ) is the inverse document frequency of keyword q i and computed as follows:…”