Online contract labor portals (i.e., crowdsourcing) have recently emerged as attractive alternatives to university participant pools for the purposes of collecting survey data for behavioral research. However, prior research has not provided a thorough examination of crowdsourced data for organizational psychology research. We found that, as compared with a traditional university participant pool, crowdsourcing respondents were older, were more ethnically diverse, and had more work experience. Additionally, the reliability of the data from the crowdsourcing sample was as good as or better than the corresponding university sample. Moreover, measurement invariance generally held across these groups. We conclude that the use of these labor portals is an efficient and appropriate alternative to a university participant pool, despite small differences in personality and socially desirable responding across the samples. The risks and advantages of crowdsourcing are outlined, and an overview of practical and ethical guidelines is provided.

No abstract

No abstract

Aim. This study investigates a novel technique for measuring video game engagement by capturing behavioral data with little task interference. Background. Flow Theory and Cognitive Load Theory provide insight into understanding engagement by analyzing the interactions between skill and task challenge. The development of this real-time measurement of engagement provides a more precise diagnostic method for designing challenging, yet cognitively engaging, tasks. Method. Flow Theory guided the design of three conditions (Boredom, Flow, and Frustration) for a video game played by 156 participants. The authors tested a potential measure of engagement based on the number of times a participant clicked a game-clock during gameplay and during intermissions, along with performance and workload data. We differentiated the three conditions by synthesizing game-clock clicks during gameplay, during intermission, and overall cognitive load. Results. Boredom showed lower cognitive load than Flow and Frustration. Frustration had significantly lower game-clock clicks during gameplay and significantly higher clicks during intermission than Flow or Boredom. Conclusion. This study’s measurement approach could potentially be used to measure cognitive and affective elements of engagement, helping to identify where in a task a person may reach a point of disengagement, and where they may choose to reengage.

No abstract