Empirical Evaluations with Domain Experts

From word clouds to Word Rain: Revisiting the classic word cloud to visualize climate change texts

“…Evaluation/validation of various types of contributions in information visualization and visual analytics is an open challenge, discussed, among others, by Lam et al 70 and Isenberg et al 71 As we have chosen the topic of climate change as the main topic for selecting the source text data to be processed, represented, and analyzed with the Word Rain technique in this study, including the case studies with IPCC reports and Nature editorials discussed above, we chose the domain expert review 72,73 as the main evaluation approach. The motivation for this choice is severalfold: (i) researchers who may actually use the Word Rain technique for representing and analyzing text data as part of their own research present an important target audience for us; (ii) such researchers most likely have at least passing knowledge/familiarity with the classic word/tag clouds, including the potential use of the technique in their field or their own contributions; (iii) the insights reported by the domain experts can complement some of our case studies and examples; and (iv) domain expert reviews are a viable option when focusing on the qualitative evaluation results (as we do not claim to provide quantitative evidence of the technique’s superiority with respect to effectiveness or efficiency for particular tasks, for instance).…”