Automatic analysis of caregiver input and child production

Abstract

The present study explores the applicability of Natural Language Processing (NLP) techniques to investigate child corpora in Korean. We employ caregiver input and child production data in the CHILDES database, currently the largest and open-access Korean child corpus data, and apply NLP techniques to the data in two ways: automatic Part-of-Speech tagging by adapting a machine learning algorithm, and (semi-)automatic extraction of constructional patterns expressing a transitive event (active transitive and suffixal passive). As the first empirical report on NLP-assisted analysis of Korean child corpora, this study is expected to reveal its advantages and drawbacks, thereby opening the window to furthering corpus-mediated research on child language development in Korean. Implications of this study’s findings will also contribute to research practice regarding developmental studies on Korean through child corpora, ensuring the reproducibility of procedures and results, which is often lacking in previous corpus-based research on child language development in Korean.

Explainability of neural networks for child language: Agent‐First strategy in comprehension of Korean active transitive construction

“…Along with the general nature of caregiver input, the models may thus have been affected by the specific word order and/or the presence of case markers in conducting the classification, particularly as shown with the two‐argument case‐marked scrambled condition (N ACC N NOM V) and the case‐less conditions (N CASE N CASE V; N CASE V). This aligns with previous reports on language‐specific challenges for automatic processing of Korean (Kim et al., 2007; Shin, 2022a). Since we are not aware of any study on language‐specific properties and NNs’ performance on child language, this claim awaits further examination.…”

“…One is that the number of first‐noun‐as‐agent patterns (3049 instances) did not exceed that of first‐noun‐as‐theme patterns (3579 instances). The other property is that the number of nominative‐first patterns (overtly marked with the nominative case marker; 3369 instances) outnumbered that of accusative‐first patterns (overtly marked with the accusative case marker; 1989 instances) despite the generally higher omission rate of the accusative case marker than that of the nominative case marker in caregiver input (Shin, 2022a). Given these characteristics, as epoch progressed, the two models may have attended primarily to the form of a specific case marker (overtly attested in a test stimulus) rather than to the meaning/function (i.e., thematic roles) of the initial noun, possibly leading to both success in one‐argument conditions where consideration of thematic role ordering was not required but partial success in the two‐argument conditions where thematic role ordering between the two arguments should be considered.…”

“…Another possible contributing factor to the model performance is around language‐specific properties. Whereas Korean caregiver input joins the general characteristics of child‐directed speech (Shin, 2022a; cf. Cameron‐Faulkner et al., 2003; Stoll et al., 2009), it also manifests language‐specific properties such as scrambling and omission of sentential components (see Appendix for the constructional‐pattern‐wise variability).…”

“…For the binary classification of test items ( Agent‐First ; Theme‐First ), these models were further trained with instances of all the constructional patterns expressing a transitive event—active transitive and suffixal passive, with scrambling and varying degrees of omission manifested—with labels indicating whether the thematic‐role ordering of these instances followed agent‐first or theme‐first (Appendix ). The instances were extracted from the pre‐processed caregiver‐input data through an automatic search process developed by Shin (2022a); every sentence for each extraction was also checked manually to ensure its accuracy. Although the focus concerning the Agent‐First strategy in this study was the active transitive, we included the suffixal passive, another major clause‐level device expressing a transitive event and the representative type of passive that children are likely to encounter in caregiver input (Shin & Deen, 2023).…”

“…In the present study, we train each NN model by patching caregiver‐input data in CHILDES (MacWhinney, 2000) onto the respective pre‐trained models. Caregiver input—which notably differ from adult language usage in terms of clausal composition (e.g., non‐human agents, partial utterances) and mode of delivery (e.g., simple, short, repetitive) (Cameron‐Faulkner et al., 2003; Shin, 2022a; Stoll et al., 2009)—is known to effectively support children's development of linguistic knowledge (Behrens, 2006; Choi, 1999; Snow, 1972). If NNs faithfully exploit this characteristic for their learning, the models in this study should approximate the children's response patterns measured by Shin (2021), showing reasonable accuracy, like their successful performance in some adult language features (Hawkins et al., 2020; Marvin & Linzen, 2019; Warstadt & Bowman, 2020; Warstadt et al., 2019).…”