How the crowd impacts commercial applications: a user-oriented approach

Abstract

As crowdsourcing has been applied to a variety of disciplines, e.g. marketing and operationalization, more and more scientists turn their sights to how the crowd innovate software engineering to produce high quality software. However, they mainly focus on the impacts brought by domain experts or experienced developers on developing and managing open source softwares, whereas how softwares are influenced by the ordinary people e.g. end users is seldom discussed and easily omitted. To fill up the research gaps, we investigate into commercial application improvement paradigm with assistance of user crowd. The approach focuses on end users by proposing a workflow loop to form a healthy cycle between them and applications. Especially, the approach propose a suggestion model to encourage users to participate into application runtime adaptation. So far, a prototype is developed to enable the crowd to raise and modify their advices, and our prior work has proven the effectiveness in which applications consider the users' advices to adapt themselves.

A survey of the use of crowdsourcing in software engineering

“…A series of studies have investigated the potential of crowdsourced software evolution and maintenance [19], [24], [29], [97], [98], [175], [178], [179], [183]- [188].…”

“…Also, evaluation of the proposed framework was performed on a socially adaptive messenger system. He et al [183] proposed a 'suggestion model' to encourage crowd users to become more closely involved in commercial software runtime adaptation. A prototype and several adaptation strategies were introduced in this study.…”

Given Enough Eyeballs, all Bugs are Shallow - A Literature Review for the Use of Crowdsourcing in Software Testing

“…Table 1 depicts the references identified per type of testing. [3], [43], [44], [45], [46], [47], [48] Non-Functional Testing [32], [43], [44], (performance); [49] (vulnerability); [50] (privacy) Validation and Acceptance Testing [11], [34], [35], [36], Usability Testing/ User Experience [35], [41], [51], [52], [53], [54], [55], [56], [57] Quality of Experience [26], [58], [59], [60], [61], [62] Research in functional and verification testing demonstrated that even complex testing tasks such as the verification of cross-browser issues [46] or the reproduction of context-sensitive app crashes [45] are possible to be tested with the crowd. In this vein, also non-functional testing such as performance testing [32] is possible.…”

“…Research in this area has shown that usability testing with a crowd is feasible, produces reliable and high quality results [11,56], is cost efficient [53,56], and can help reduce critical usability testing obstacles such as resource constraints, limited understanding of the usability concept, and resistance to the adoption of usability practices [51]. In addition to that, scholars have developed several workflows and corresponding tools to use crowdsourced usability testing [35,54,55], including machine learning approaches to group similar responses and filter meaningless submissions to reduce workload for test managers [64].…”