Crowdsourcing platform for collaboration management in vulnerability verification

Given Enough Eyeballs, all Bugs are Shallow - A Literature Review for the Use of Crowdsourcing in Software Testing

“…Most papers are published in IEEE or ACM affiliated conferences, hence it is not of surprise that many of the papers (3) develop prototypes (e.g., [49], [37]) or attempt to optimize parts of the crowdsourcing process with algorithms (e.g., test report prioritization [48]) to increase the feasibility and cost efficiency of CST. However, these solutions are practical examples and the underlying design principles remain unknown.…”

“…Table 1 depicts the references identified per type of testing. [3], [43], [44], [45], [46], [47], [48] Non-Functional Testing [32], [43], [44], (performance); [49] (vulnerability); [50] (privacy) Validation and Acceptance Testing [11], [34], [35], [36], Usability Testing/ User Experience [35], [41], [51], [52], [53], [54], [55], [56], [57] Quality of Experience [26], [58], [59], [60], [61], [62] Research in functional and verification testing demonstrated that even complex testing tasks such as the verification of cross-browser issues [46] or the reproduction of context-sensitive app crashes [45] are possible to be tested with the crowd. In this vein, also non-functional testing such as performance testing [32] is possible.…”

Web Science Challenges in Researching Bug Bounties

“…Despite the increasing popularity of bug bounties [6], and their seeming relationship with crowdsourcing, we were unaware of any work which considered bug bounties within the context of crowdsourcing. The one exception to this was Su & Pan, who proposed a system to introduce microtasking to the process, where additional actors would test and verify the vulnerability submitted by another researcher [36].…”

“…It is possible to do these in a 'microtask' scenario, but with a 'macrotask' it is not possible to break down the tasks into anything smaller [8,35], requiring a di erent set of criteria for consideration. Areas such as validation of reports represent an obvious area where a bug bounty could include additional tasks [36], which are more akin to microtask crowdsourcing where a consensus would be desirable. However, there are obvious security risks with this, so great care would be required before any practical implementation.…”

“…The work described earlier by Su & Pan about adding falls into this category speci cally relating to bug bounties of adding a microtask for veri cation [36]. This could additionally be considered as part of task decomposition, but it is better to consider it here, since it is an additional task rather than decomposing the vulnerability discovery.…”