Abstract-Video quality assessment with subjective testing is both time consuming and expensive. An interesting new approach to traditional testing is the so-called crowdsourcing, moving the testing effort into the internet. We therefore propose in this contribution the QualityCrowd framework to effortlessly perform subjective quality assessment with crowdsourcing. QualityCrowd allows codec independent quality assessment with a simple web interface, usable with common web browsers. We compared the results from an online subjective test using this framework with the results from a test in a standardized environment. This comparison shows that QualityCrowd delivers equivalent results within the acceptable inter-lab correlation. While we only consider video quality in this contribution, QualityCrowd can also be used for multimodal quality assessment.

Abstract-Video quality evaluation with subjective testing is both time consuming and expensive. An interesting new approach to traditional testing is the so-called crowdsourcing, moving the testing effort into the internet. The QualityCrowd framework allows codec independent, crowd-based video quality assessment with a simple web interface, usable with common web browsers. However, due to its codec independent approach, the framework can pose high bandwidth requirements on the coordinating server. We therefore propose in this contribution a cloud-based extension of the QualityCrowd framework in order to perform subjective quality evaluation as a cloud application. Moreover, this allows us to access an even larger pool of potential participants due to the improved connectivity. We compare the results from an online subjective test using this framework with the results from a test in a standardized environment. This comparison shows that QualityCrowd delivers equivalent results within the acceptable inter-lab correlation.

No abstract

No abstract

No abstract