Computer Generation of Semantic Differential (SD) Questionnaires

Abstract

EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 1969, 29, 191-192. THE typical SD questionnaire consists of directions to S followed by the first concept with its associated adjectival scales. Realizing the possibility that progressive effects and treatment interactions may occur, E determines the order and polarity of the scales in some random fashion. Thus, while the ordering of the scales and their polarity remains invariant, at least the effects are free of E's bias with respect to the variables being studied.Order effects among the concepts included are often &dquo;balanced&dquo; by presenting the concepts to Ss in several different randomly determined orders. The solution, then, has been to take account of, but not necessarily minimize, proximity error within SD questionnaires.While the weaknesses of this solution are obvious, better solutions have not been economically feasible. Print shop and office duplicating machines are not generally designed to produce variable formats. Moreover, how can E be sure he is reducing substantially these kinds of bias by having a number of different versions of SD questionnaire sheets used in his data collection?Each of these objections now may be rebutted. Houston (1967) has reported improved methods of controlling proximity error by employing a Monte Carlo technique for identifying Latin squares whose columns yield a sequence of permutations of n items such that proximity error may be dramatically reduced for n < 23. Since many applications of the SD involve fewer than 23 scales per concept and fewer than 23 concepts per administration, Houston's work has obvious import for Es using the SD.

State-Dependence Effects in Surveys

“…Indirectly, these papers also shed light on state dependence, as a pattern of similar responses produces high (but potentially spurious) reliability. Finally, the survey literature has considered "proximity error" (Guilford 1954, Kane 1969, which occurs when adjacent attributes are evaluated more similarly relative to attributes located farther apart. Proximity error has been mainly studied in the employee appraisal research (Zedeck and Blood 1974) and is close in concept to the state-dependence effect examined here.…”

“…An intriguing question is whether a systematic response tendency in the form of state dependence is present in survey responses. The notion of state dependence is related to previously studied concepts of "response carryover" (e.g., Bickart 1993, Tourangeau et al 2000, "nondifferentiation" (Krosnick 1991), "patterned" responses and "proximity error" (e.g., Guilford 1954, Kane 1969, Yamamoto 1995, Zedeck and Blood 1974, as well as to the research on common stimulus material as in the testlet theory Wainer et al 2007;Wang et al 2002Wang et al , 2010. Our research deviates from much of this literature by focusing on carryover resulting from the response to the previous questions as opposed to because of the question content or common structure.…”