The Effect of Different Cultural Lenses on Reliability and Validity in Observational Data: The Example of Chinese Immigrant Parent–Toddler Dinner Interactions

Abstract

This study used a mixed methodology to investigate reliability, validity, and analysis level with Chinese immigrant observational data. European-American and Chinese coders quantitatively rated 755 minutes of Chinese immigrant parent-toddler dinner interactions on parental sensitivity, intrusiveness, detachment, negative affect, positive affect, and parents' confidence. They also gave qualitative coding rationales for their quantitative ratings. Analysis of quantitative ratings indicated systematic differences associated with coder ethnicity. Such differences could threaten the reliability between coders with different ethnicities. Analysis of qualitative rationales revealed different cultural links between concepts and behavior cues, which could challenge the crosscultural validity of a macro-level coding system. When a coding system is incongruous with coders' cultural schemata, even after intensive training, the effect of culture cannot be eliminated. Implications of these findings for observational cultural research are discussed.

Observed Differences in Social Behaviors Exhibited in Peer Interactions Between Youth With Spina Bifida and Their Peers: Neuropsychological Correlates

“…Research also suggests that interracial friendships tend to be less stable and lower in quality (Graham, Taylor, & Ho, 2009). Although such differences may be due to cultural and gender mechanisms, the coders' racial and gender biases also may have contributed to these findings (Wang, Wiley, & Zhou, 2007). Codes assigned to the peer interactions in the present study may reflect a female and White perspective given the characteristics of the coders in this study.…”

Detached Parenting and Toddler Problem Behavior in Early Head Start Families

“…Examination of extreme detached behaviors, as in child neglect, would contribute to such efforts. There also is a need to address the influence of culture on the conceptualization, operationalization, and assessment of detached parenting because parenting behaviors may have different meanings across cultures (Wang, Wiley, & Zhou, ).…”

Independence-supportive praise versus interdependence-promoting praise

“…For this reason, an ethnicity/language-matched coding approach was used in the present study (Wang, Wiley, & Zhou, 2007). Two Chinese-immigrant coders (both came to the USA before their fifth birthday) coded the Chinese-immigrant video data, while two European-American coders coded the EuropeanAmerican video data.…”