Choosing Units of Analysis in Temporal Discourse

Barany, Amanda; Phillips, Michael; Kawakubo, Anthony J. Taiki; Oshima, Jun

doi:10.1007/978-3-030-93859-8_6

Cited by 6 publications

(9 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For this purpose, we employ a dataset that comprises transcripts of press releases and public addresses delivered by governmental leaders from seven countries from late February to late March 2020 [17]. The authors used a codebook incorporating seven categories: Medical Positive, Medical Negative, Economic Positive, Economic Negative, Social Positive, Social Negative, and Political Positive.…”

Section: Study Contextsmentioning

confidence: 99%

“…The average inter-rater agreement, measured by Cohen's Kappa, was 0.94. For a more comprehensive discussion of the codebook, refer to [17].…”

Section: Study Contextsmentioning

confidence: 99%

“…Following the process outlined by Cai et al [5], we used an iterative approach to craft regular expressions in nCoder that maximized agreement between the classifier and human coders for each code category. We first established a set of regular expressions based on the construct definitions provided by [17] and applied them to code the training set. We then analyzed any instances where nCoder and human coders disagreed and evaluated the effect of adding or deleting regular expressions on the overall kappa value.…”

Section: Coding Process With Ncoder and Chatgptmentioning

confidence: 99%

“…We adopted a similar approach for ChatGPT. For each construct, we provided ChatGPT with the construct name and original definition that was used by the authors [17]. We then addressed each disagreement between ChatGPT and human coders by requesting explanations from ChatGPT regarding its decisions.…”

Section: Coding Process With Ncoder and Chatgptmentioning

confidence: 99%

“…Given these possibilities, this work is an exploratory look at the potential of ChatGPT for supporting data coding processes in Quantitative Ethnography. We evaluate the accuracy and reliability of ChatGPT within the context of a dataset consisting of press releases and public addresses delivered by governmental leaders from seven countries during the period of late February to late March 2020 [17]. We investigate the consistency and construct validity of both ChatGPT and nCoder's automated coding procedures as compared to human-generated codes, and examine the role that ChatGPT's explanations of its coding decisions in supporting humans in revising a coding scheme.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

From nCoder to ChatGPT: From Automated Coding to Refining Human Coding

Zambrano,

Liu,

Barany

et al. 2023

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

This paper investigates the potential of utilizing ChatGPT (GPT-4) as a tool for supporting coding processes for Quantitative Ethnography research. We compare the use of ChatGPT and nCoder, the most widely used automated coding tool in the QE community, on a dataset of press releases and public addresses delivered by governmental leaders from seven countries from late February to late March 2020. The study assesses the accuracy of the automated coding procedures between the two tools, and the role that ChatGPT's explanations of its coding decisions can play in improving the consistency and construct validity of human-generated codes. Results suggest that both ChatGPT and nCoder have advantages and disadvantages depending on the context, nature of the data, and researchers' goals. While nCoder is useful for straightforward coding schemes represented through regular expressions, ChatGPT can better capture a variety of language structures. ChatGPT's ability to provide explanations for its decisions can also help enhance construct validity, identify ambiguity in code definitions, and assist human coders in achieving high interrater reliability. Although we identify limitations of ChatGPT in coding constructs open to human interpretations and encompassing multiple concepts, we highlight opportunities and potential benefits provided by ChatGPT as a tool to support human researchers in their coding process.

show abstract

Section: Study Contextsmentioning

confidence: 99%

“…The average inter-rater agreement, measured by Cohen's Kappa, was 0.94. For a more comprehensive discussion of the codebook, refer to [17].…”

Section: Study Contextsmentioning

confidence: 99%

Section: Coding Process With Ncoder and Chatgptmentioning

confidence: 99%

Section: Coding Process With Ncoder and Chatgptmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

From nCoder to ChatGPT: From Automated Coding to Refining Human Coding

Zambrano,

Liu,

Barany

et al. 2023

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

show abstract

Segmentation and Code Co-occurrence Accumulation: Operationalizing Relational Context with Stanza Windows

Zörgő

2023

Communications in Computer and Information Science

View full text Add to dashboard Cite

Depending on analytical goals and techniques, qualitative data may be coded and segmented to investigate code or code co-occurrence frequencies. As codes are relevant aspects of data vis-à-vis the topic of inquiry, segments are meaningful divisions of those data. To explore various modes of segmentation, their underlying assumptions, and effects on potential models, the framework and terminology of Epistemic Network Analysis was employed as an analytical tool where coding and segmentation both contribute to data visualization. Three operationalizations of segmentation are elaborated: moving, infinite, and whole conversation stanza windows and demonstrated through instances where each of these may be applicable to data.

show abstract

An Examination of Student Loan Borrowers’ Attitudes Toward Debt Before and During COVID-19

Bright

Barany

2023

Communications in Computer and Information Science

View full text Add to dashboard Cite

Choosing Units of Analysis in Temporal Discourse

Cited by 6 publications

References 15 publications

From nCoder to ChatGPT: From Automated Coding to Refining Human Coding

From nCoder to ChatGPT: From Automated Coding to Refining Human Coding

Segmentation and Code Co-occurrence Accumulation: Operationalizing Relational Context with Stanza Windows

An Examination of Student Loan Borrowers’ Attitudes Toward Debt Before and During COVID-19

Contact Info

Product

Resources

About