Methodology in the Mirror: A Living, Systematic Review of Works in Quantitative Ethnography

Abstract

Quantitative Ethnography is a nascent field now formulating the specifics of its conceptual framework and terminology for a unified, quantitativequalitative methodology. Our living, systematic review aims to shed light on decisions in research design that the community has made thus far in the domain of data collection, coding & segmentation, analysis, and how Quantitative Ethnography as a methodology is conceptualized. Our analysis intends to spur discussions on these issues within the community and help establish a lingua franca.

Short essay: Current issues in digital ethnography: is all ethnography digital?

“…On one hand, this is an interesting direction because, as Hammersley (2018) suggests, ethnography is an approach that encourages possible methods to be used to produce a holistic understanding of cultural practices. On the other hand, considering qualitative associations of ethnography and research challenges that statistical or digital data will provide an objective picture of daily practices is still up for debate (Kavanough and Maratea, 2020;Z€ org} o et al, 2021;Lavorgna and Holt, 2021;Lavorgna and Sugiura, 2022).…”

From nCoder to ChatGPT: From Automated Coding to Refining Human Coding

“…Establishing inter-coder agreement can reduce the time and effort spent by allowing researchers to code sections of the dataset independently but remain time-consuming. While manual coding techniques remain popular in both QE and broader research [3], such approaches remain unsuitable for larger datasets, and the potential for human error remains.…”

“…Once researchers deem the agreement between themselves and the detector sufficiently high, based on indicators such as Kappa, precision, recall, and Shaffer's rho [1,6,7], researchers can then generalize these classifiers to code the entire dataset. While nCoder has proved useful in QE research [3], there are still concerns that nCoder may be unable to capture all cases where a human would identify a code (recall), as regular expressions may not match human coders' capacity to consider contextual features or semantic nuance.…”

Learning experience network analysis for design-based research

“…What we develop in this paper is a novel approach to contribute to the DBR toolkit by using a new variation of network analysis to compliment others in the DBR toolkit such as ENA. While quantitative ethnography methods (Zörgő et al , 2022) such as ENA excel at helping researchers understand epistemic aspects of learning experiences such as identifying and describing epistemic frames, comparison of epistemic frames across people, groups or contexts and tracking of changes in epistemic frames over time (Arastoopour Irgens and Eagan, 2023; Zörgő et al , 2022), our method seeks to help researchers understand ontological aspects such as how the design of learning is experienced. In other words, ENA is most powerful when we want to understand learning itself, where our method is useful when we want to understand the complex patterns of interdependencies between various experiences of learners engaging in a designed learning process.…”