How Reliable is the Crowdsourced Knowledge of Security Implementation?

Abstract

Stack Overflow (SO) is the most popular online Q&A site for developers to share their expertise in solving programming issues. Given multiple answers to certain questions, developers may take the accepted answer, the answer from a person with high reputation, or the one frequently suggested. However, researchers recently observed that SO contains exploitable security vulnerabilities in the suggested code of popular answers, which found their way into security-sensitive highprofile applications that millions of users install every day. This observation inspires us to explore the following questions: How much can we trust the security implementation suggestions on SO? If suggested answers are vulnerable, can developers rely on the community's dynamics to infer the vulnerability and identify a secure counterpart?To answer these highly important questions, we conducted a comprehensive study on security-related SO posts by contrasting secure and insecure advice with the community-given content evaluation. Thereby, we investigated whether SO's gamification approach on incentivizing users is effective in improving security properties of distributed code examples. Moreover, we traced the distribution of duplicated samples over given answers to test whether the community behavior facilitates or prevents propagation of secure and insecure code suggestions within SO.We compiled 953 different groups of similar security-related code examples and labeled their security, identifying 785 secure answer posts and 644 insecure answer posts. Compared with secure suggestions, insecure ones had higher view counts (36,508 vs. 18,713), received a higher score (14 vs. 5), and had significantly more duplicates (3.8 vs. 3.0) on average. 34% of the posts provided by highly reputable so-called trusted users were insecure.Our findings show that based on the distribution of secure and insecure code on SO, users being laymen in security rely on additional advice and guidance. However, the communitygiven feedback does not allow differentiating secure from insecure choices. The reputation mechanism fails in indicating trustworthy users with respect to security questions, ultimately leaving other users wandering around alone in a software security minefield.Index Terms-Stack Overflow, crowdsourced knowledge, social dynamics, security implementation K e y P a i r G e n e r a t o r kpg = K e y P a i r G e n e r a t o r . g e t I n s t a n c e ( " RSA " ) ; kpg . i n i t i a l i z e ( 1 0 2 4 ) ; K e y P a i r kp = kpg . g e n e r a t e K e y P a i r ( ) ; RSAPublicKey pub = ( RSAPublicKey ) kp . g e t P u b l i c ( ) ; RSAPrivateKey p r i v = ( RSAPrivateKey ) kp . g e t P r i v a t e ( ) ;Hash: In the context of password-based key derivation, digital signatures, and authentication/authorization, developers may explicitly invoke broken hash functions. Listing 4 shows an example using MD5.

Cybersecurity Education and Formal Methods

“…Education Could certainly do better [48], though there are encouraging signs [49] and useful ideas when it comes to improving informal resources [59]. However, informal resources can be dangerous when it comes to security, and [49] recommends giving all students the advice in [60]: "If you pick up a SSL/TLS answer from Stack Overflow, there's a 70% chance it's insecure".…”

Understanding performance concerns in the API documentation of data science libraries

“…For example, Zhang et al found that one third of SO posts contain potential API misuses [47]. Chen et al found that a large proportion of security implementations on SO is insecure, and that the corresponding posts have higher scores and more duplicates compared to posts with secure suggestions [15]. In the future, we also plan to investigate the quality of unocial performance-related information from crowdsourced data.…”

On the use of C# Unsafe Code Context

“…Based on best of our knowledge, this is the first study on security of C# codes. several studies investigated security related issues in languages like Java and Python [22], [11], [5]. However, as mentioned no study has focused on security of C# codes and thus no study on C# code snippets in community question and answer websites exist.…”