Human versus computer-aided qualitative data analysis ratings: Spiritual content in dream reports and diary entries.

Abstract

Three computer-aided qualitative data analysis software (CAQDAS) were compared to each other and to a hand-coded method for analyzing spiritual content in 49 dream journals and 11 diary entries. The CAQDAS ratings demonstrated strong convergence with each other, despite using different assumptions; the hand-coded ratings were less correlated with the CAQDAS than the CAQDAS were to each other. The pattern of convergence and divergence of the ratings (hand coded vs. CAQDAS) and traits (dream vs. diary) was compared in a multitrait multimethod matrix, interpreted as supportive of the validity of both methods. However, all three CAQDAS uncovered material missed by the hand-coded method, suggesting their relative advantage, even when applied to content as ambiguous as spiritual categories.John Henry was a folklore figure of the 1800s who boasted he was the best at driving railroad spikes by hand, so he was pitted against a newly created steam-driven hammer. In that legendary race of human brawn versus machine, he prevailed but died immediately thereafter of exhaustion (Nelson, 2008). This tale has become emblematic of the escalating struggle between human pride and the fact that machines can outdo humans in an increasing number of tasks. No one would now claim that humans can prevail in a road race against a car or in a calculating contest against a computer. But there are those who might claim that humans can still prevail in some areas, such as making meaningful patterns out of ambiguous qualitative data. The burgeoning field of data aggregation, exemplified by those increasingly annoying pop-ups during computer searches that seem to read our deepest thoughts by instantly tailoring advertisements from combing through recent e-mails and Web searches, portends the future that perhaps even the analysis of qualitative data might completely yield to the machine. Yesterday's news of Deep Blue, the supercomputer that could beat any human at chess, is now augmented by the astounding feat of Watson, the next This paper is partially based on the fi rst author's dissertation (Marshall, 2005), supervised by Friedman, and additional details on the study can be found there.