Modeling problem difficulty and expertise in stackoverflow

Hanrahan, Benjamin V.; Convertino, Gregorio; Nelson, Les

doi:10.1145/2141512.2141550

Cited by 73 publications

(42 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, it suggests the need to examine whether present methods developed for identifying highly active experts (e.g. [21,19,24,12]) can accurately recognize the experts in our results. Second, it seems promising to use task allocation mechanisms to direct experts to primarily answer difficult questions when they contribute to the system.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Contributor profiles, their dynamics, and their importance in five q&a sites

Furtado

Andrade

Oliveira

et al. 2013

Proceedings of the 2013 Conference on Computer Supported Cooperative Work

View full text Add to dashboard Cite

Q&A sites currently enable large numbers of contributors to collectively build valuable knowledge bases. Naturally, these sites are the product of contributors acting in different ways -creating questions, answers or comments and voting in these -, contributing in diverse amounts, and creating content of varying quality. This paper advances present knowledge about Q&A sites using a multifaceted view of contributors that accounts for diversity of behavior, motivation and expertise to characterize their profiles in five sites. This characterization resulted in the definition of ten behavioral profiles that group users according to the quality and quantity of their contributions. Using these profiles, we find that the five sites have remarkably similar distributions of contributor profiles. We also conduct a longitudinal study of contributor profiles in one of the sites, identifying common profile transitions, and finding that although users change profiles with some frequency, the site composition is mostly stable over time.

show abstract

Section: Discussionmentioning

confidence: 99%

“…For example, Riahi et al [24] and Hamrahan et al [12] explore automatic means for identifying the most adequate experts for a question. Pal et al showed that it is possible to predict which users will be highly active experts using data from their first weeks of activity [21,19].…”

Section: Examining or Identifying Experts And Elite Contributorsmentioning

confidence: 99%

Contributor profiles, their dynamics, and their importance in five q&a sites

Furtado

Andrade

Oliveira

et al. 2013

Proceedings of the 2013 Conference on Computer Supported Cooperative Work

View full text Add to dashboard Cite

show abstract

“…Hanrahan et al present a summary perspective of the distribution of reputation in Stack Exchange and report that more than half of the users have a reputation score of less than 10 and answer fewer questions than they ask [16]. Using answer latency as a proxy for question difficulty, no significant relationship was found between user expertise and question duration, however, particularly difficult questions are often answered by the questioner to a significant degree.…”

Section: Related Workmentioning

confidence: 99%

Mining Stack Exchange: Expertise Is Evident from Initial Contributions

Posnett

Warburg

Dévanbu

et al. 2012

2012 International Conference on Social Informatics

View full text Add to dashboard Cite

Stack Exchange is a very popular Question & Answer internet community. Users can post questions on a wide variety of topics; other users provide answers, usually within minutes. Participants are not compensated for their services and anyone can freely gain value from the efforts of the users; Stack Exchange is therefore a gift economy. Users, however, do gain reputation points when other users "upvote" their questions and/or answers. Stack Exchange thus functions as a learning community with a strong reputation-seeking element that creates a valuable public good, viz., the Q&A archive. The incentive structure of the community suggests that over time, the quality of the product (viz.., delivered answers) steadily improves, and furthermore, that any individual who durably participates in this community for an extended period also would enjoy an increase in the quality of their output (viz., the answers they provide). We investigate the validity of these widely held beliefs in greater detail, using data downloaded from Stack Exchange. Our analysis indicates that these intuitions are actually not supported by the data; indeed the data suggests that overall answer scores decrease, and that people's tenure with the community is unrelated to the quality of their answers. Most interestingly, we show that answering skill, i.e. getting high average answer scores, which is different than reputation, is evident from the start and persists during one's tenure with the community. Conversely, people providing low rated answers are likely to have done so from the start.

show abstract

“…A considerable body of research devoted to the study of StackOverflow.com has developed recently to investigate such diverse topics as how the 'gaming' elements incentivize contributions [4], exploring the 'activeness' of contributors [5], and the automated identification of topical 'experts' [6] [7]. At the present, however, the subject of how users (and novices in particular) perceive answer quality has received little attention.…”

Section: Introductionmentioning

confidence: 99%

Perceptions of answer quality in an online technical question and answer forum

Hart

Sarma

2014

Proceedings of the 7th International Workshop on Cooperative and Human Aspects of Software Engineering

View full text Add to dashboard Cite

Software developers are used to seeking information from authoritative texts, such as a technical manuals, or from experts with whom they are familiar. Increasingly, developers seek information in online question and answer forums, where the quality of the information is variable. To a novice, it may be challenging to filter good information from bad. Stack Overflow is a Q&A forum that introduces a social reputation element: users rate the quality of posted answers, and answerers can accrue points and rewards for writing answers that are rated highly by their peers. A user that consistently authors good answers will develop a good 'reputation' as recorded by these points. While this system was designed with the intent to incentivize high-quality answers, it has been suggested that information seekers-and particularly technical novices-may rely on the social reputation of the answerer as a proxy for answer quality. In this paper, we investigate the role that this social factor-as well as other answer characteristics-plays in the information filtering process of technical novices in the context of Stack Overflow. The results of our survey conducted on Amazon.com's Mechanical Turk indicate that technical novices assess information quality based on the intrinsic qualities of the answer, such as presentation and content, suggesting that novices are wary to rely on social cues in the Q&A context.

show abstract

Modeling problem difficulty and expertise in stackoverflow

Cited by 73 publications

References 4 publications

Contributor profiles, their dynamics, and their importance in five q&a sites

Contributor profiles, their dynamics, and their importance in five q&a sites

Mining Stack Exchange: Expertise Is Evident from Initial Contributions

Perceptions of answer quality in an online technical question and answer forum

Contact Info

Product

Resources

About