Crowdsourced Feedback With Imagery Rather Than Text

Robb, David A.; Padilla, Stefano; Kalkreuter, Britta; Chantler, Mike J.

doi:10.1145/2702123.2702470

Cited by 14 publications

(15 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Essentially an open call for labor with flexible contractual arrangements [47], crowdsourcing provides a convenient, low-cost solution for obtaining specific feedback that is arguably objective and valid [38,69]. Crowdsourcing is already being used to generate annotations for images depicting human behavior with reported success [27,53,54].…”

Section: Crowdsourcing Emotion Judgmentsmentioning

confidence: 99%

Can the crowd tell how I feel? Trait empathy and ethnic background in a visual pain judgment task

Matsangidou

Otterbacher

Ang

et al. 2018

Univ Access Inf Soc

View full text Add to dashboard Cite

Many advocate for artificial agents to be empathic. Crowdsourcing could help, by facilitating human-in-the-loop approaches and data set creation for visual emotion recognition algorithms. Although crowdsourcing has been employed successfully for a range of tasks, it is not clear how effective crowdsourcing is when the task involves subjective rating of emotions. We examined relationships between demographics, empathy, and ethnic identity in pain emotion recognition tasks. Amazon MTurkers viewed images of strangers in painful settings, and tagged subjects' emotions. They rated their level of pain arousal and confidence in their responses, and completed tests to gauge trait empathy and ethnic identity. We found that Caucasian participants were less confident than others, even when viewing other Caucasians in pain. Gender correlated to word choices for describing images, though not to pain arousal or confidence. The results underscore the need for verified information on crowdworkers, to harness diversity effectively for metadata generation tasks.

show abstract

Section: Crowdsourcing Emotion Judgmentsmentioning

confidence: 99%

Can the crowd tell how I feel? Trait empathy and ethnic background in a visual pain judgment task

Matsangidou

Otterbacher

Ang

et al. 2018

Univ Access Inf Soc

View full text Add to dashboard Cite

show abstract

“…According to Jakobsen's model of communication [19] a message can have its own inherent artistic quality. The fact that designers shown image-based feedback by Robb et al [47] were inspired to make changes but were not so inspired by text feedback bears this out. Secondly, if designers can build a following by engaging people in feedback, the content of each message in the conversation need not be crucial.…”

Section: Feedback-givers Valuing Engagement Over Claritymentioning

confidence: 99%

“…The image-based emotion feedback method (IEFM) was developed to provide fashion and interior designers with visual feedback on the perceived mood of their designs. It was found in an evaluation to be popular with the designers receiving the feedback [47]. Those providing the feedback choose images from perceptually organized image browsers instead of using text.…”

Section: Introductionmentioning

confidence: 99%

Image-based Emotion Feedback

Robb

Padilla

Methven

et al. 2017

Proceedings of the 2017 Conference on Designing Interactive Systems

Self Cite

View full text Add to dashboard Cite

In previous work we developed a method for interior designers to receive image-based feedback about a crowd's emotions when viewing their designs. Although the designers clearly desired a service which provided the new style of feedback, we wanted to find out if an internet crowd would enjoy, and become engaged in, giving emotion feedback this way. In this paper, through a mixed methods study, we expose whether and why internet users enjoy giving emotion feedback using images compared to responding with text. We measured the participants' cognitive styles and found that they correlate with the reported utility and engagement of using images. Those more visual than they are verbal were more engaged by using images to express emotion compared to text. Enlightening qualitative insights reveal, surprisingly, that half of our participants have an appetite for expressing emotions this way, value engagement over clarity, and would use images for emotion feedback in contexts other than design feedback.

show abstract

“…Crowd feedback is particularly appealing due to its scalability and availability outside of classroom or studio contexts. This feedback does not necessarily be textual but can also be visual [34]. Crowds are also capable of contributing diverse perspectives that may be difficult to find within a classroom [8].…”

Section: Crowdsourcing Design Feedbackmentioning

confidence: 99%

Almost an Expert

Yuan

Luther

Krause

et al. 2016

Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work &Amp; Social Computing

View full text Add to dashboard Cite

Expert feedback is valuable but hard to obtain for many designers. Online crowds can provide fast and affordable feedback, but workers may lack relevant domain knowledge and experience. Can expert rubrics address this issue and help novices provide expert-level feedback? To evaluate this, we conducted an experiment with a 2×2 factorial design. Student designers received feedback on a visual design from both experts and novices, who produced feedback using either an expert rubric or no rubric. We found that rubrics helped novice workers provide feedback that was rated nearly as valuable as expert feedback. A follow-up analysis on writing style showed that student designers found feedback most helpful when it was emotionally positive and specific, and that a rubric increased the occurrence of these characteristics in feedback. The analysis also found that expertise correlated with longer critiques, but not the other favorable characteristics. An informal evaluation indicates that experts may instead have produced value by providing clearer justifications.

show abstract

Crowdsourced Feedback With Imagery Rather Than Text

Cited by 14 publications

References 27 publications

Can the crowd tell how I feel? Trait empathy and ethnic background in a visual pain judgment task

Can the crowd tell how I feel? Trait empathy and ethnic background in a visual pain judgment task

Image-based Emotion Feedback

Almost an Expert

Contact Info

Product

Resources

About