Revisiting Grammatical Complexity in L2 Writing via Exploratory Factor Analysis

Lan, Ge; Li, Xiaorui; Zhang, Qiusi

doi:10.3389/fpsyg.2022.860753

Cited by 3 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All these reviewed L2 Chinese studies echo SLA researchers’ argument for encompassing multiple grammatical features in single studies ( Lu and Ke, 2018 ; Zhang and Tao, 2018 ) and investigating grammatical complexity from multiple dimensions, i.e., global, clausal, and phrasal levels as well as features that are sensitive to interlanguage development and conceptual demands of tasks ( Norris and Ortega, 2009 ; Robinson et al, 2009 ; Biber et al, 2011 ; Lan et al, 2022a ), for which POS serves as the basis. Several computational tools automatically processing English texts have contributed significantly to the rapid increase and advancement of studies on grammatical complexity in relation to ESL learners’ writing development/quality; however, counterpart studies in L2 Chinese have primarily relied on labor-intensive manual annotation in SLA.…”

Section: Literature Reviewmentioning

confidence: 82%

“…A further look into the only one correct tagging among a total of four such constructions shows that the period mark used immediately after de , indicating the end of the sentence overtly, may help Stanza successfully recognize de as UH. Moreover, given the increasing research interest in the role of noun phrases in writing (e.g., Lan et al, 2022a ; Lu and Wu, 2022 ; Pan, 2023 ) and L2 Chinese learners’ different developmental paths of -de in associative and modifying phrases ( Zhang, 2002 ), we suggest that Stanza distinguishes DEG and DEC as described in the Penn Chinese Treebank 3.0 to improve the identification of fine-grained linguistic features.…”

Section: Discussionmentioning

confidence: 99%

“…These measures cover five important grammatical types: length of production (e.g., mean length of clauses), subordination (e.g., clauses per T-unit), coordination (e.g., coordinate phrases per clause), sentential complexity (i.e., clauses per sentence), and particular structures (e.g., complex nominals per T-unit). The L2SCA has facilitated empirical research on grammatical complexity in ESL writing, for example, Lu (2011) , Yang et al (2015) , Yoon (2017) , Yoon and Polio (2017) , Eckstein and Ferris (2018) , Casal and Lee (2019) , and Lan et al (2022a) . Another popular computational tool that is used in register studies is Biber Tagger, developed by Biber (1988) to tag the form and function of a wide range of lexico-grammatical features (e.g., attributive adjectives, -ing clauses as post modifiers).…”

Section: Literature Reviewmentioning

confidence: 99%

See 2 more Smart Citations

Part of speech tagging of grammatical features related to L2 Chinese development: A case analysis of Stanza in the L2 writing context

Lan

Pan²,

Sun³

et al. 2023

Front. Psychol.

Self Cite

View full text Add to dashboard Cite

Grammatical complexity has received extensive attention in second language acquisition. Although computational tools have been developed to analyze grammatical complexity, most relevant studies investigated this construct in the context of English as a second language. In response to an increasing number of L2 Chinese learners, it is important to extend the investigation of grammatical complexity in L2 Chinese. To promote relevant research, we evaluated the new computational tool, Stanza, on its accuracy of part-of-speech tagging for L2 Chinese writing. We particularly focused on eight grammatical features closely related to L2 Chinese development. Then, we reported the precisions, recalls, and F-scores for the individual grammatical features and offered a qualitative analysis of systematic tagging errors. In terms of the precision, three features have high rates, over 90% (i.e., ba and bei markers, classifiers, -de as noun modifier marker). For recall, four features have high rates, over 90% (i.e., aspect markers, ba and bei markers, classifiers, -de as noun modifier marker). Overall, based on the F-scores, Stanza has a good tagging performance on ba and bei markers, classifiers, and -de as a noun modifier marker. This evaluation provides research implications for scholars who plan to use this computational tool to study L2 Chinese development in second language acquisition or applied linguistics in general.

show abstract

Section: Literature Reviewmentioning

confidence: 82%

Section: Discussionmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Part of speech tagging of grammatical features related to L2 Chinese development: A case analysis of Stanza in the L2 writing context

Lan

Pan²,

Sun³

et al. 2023

Front. Psychol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Given the complexity of linguistics, EAP writing poses significant challenges for nonnative speakers, especially in the realm of academic writing. The intricacies of linguistics encompass, but are not limited to, lexical complexity, grammatical complexity, and stylistic complexity [3]. Even a singular aspect of complexity can be deconstructed into finer granularity for study [4], yet multi-granular research has been scarce in the past.…”

Section: Related Workmentioning

confidence: 99%

“…These metrics collectively contribute to a holistic understanding of how well the Revised Draft matches the Reference across different dimensions of how well the Revised Draft matches the Reference across different dimensions of text analysis, enabling a nuanced and thorough evaluation. Specifically, we employed textstat 2 to calculate the FRE, the Natural Language Toolkit (NLTK) 3 to compute GFI and AJS, Babelfy [35] to perform WSD, nlg-eval 4 to calculate ROUGE-L, EACS, BLUE and METEOR, and a simple Python script to compute the UWR. For a comprehensive examination of the prompts utilized in this study, please refer to Appendix A.…”

Section: Metric Selectionmentioning

confidence: 99%

Exploring Sentence-level Revision Capabilities of LLMs in English for Academic Purposes Writing Assistance

Du,

Hashimoto

2024

Preprint

View full text Add to dashboard Cite

The English for Academic Purposes (EAP) is pivotal for scholarly communication; however, it poses significant challenges for non-native English speakers. Recently, Large Language Models (LLMs) have been extensively utilized in EAP to assist with writing tasks. EAP writing assistance typically encompasses several downstream tasks in natural language processing (NLP), such as Grammatical Error Correction (GEC). Non\-etheless, some studies have revealed that the performance of LLMs in GEC tasks is inferior to traditional GEC solutions. To explore the capabilities of LLMs more thoroughly in aspects like deep semantic and syntactic structures, this study aims to rigorously assess the performance of LLMs in the Sentence-level Revision (SentRev) task. We designed three sets of meticulous experiments to evaluate the efficacy of different LLMs. The first experiment assessed LLMs using prompts in ten different languages, finding that the SentRev performance of LLMs was heavily influenced by the language of the prompt and the quality of the input text. The second experiment investigated the performance of English LLMs with minimal prompting in the SentRev task, yet the results showed no significant changes, contradicting some prior studies. In the third experiment, we devised an innovative and straightforward method that significantly enhanced the performance of multiple LLMs by integrating academic phrases from the Formulaic Language (FL) Academic Phrasebank\footnote{\url{https://www.phrasebank.manchester.ac.uk/}}, thus overcoming the performance limitations imposed by different languages on LLMs. Additionally, our study highlights the deficiencies in existing evaluation benchmarks and suggests that higher-level, discourse-based EAP text evaluation benchmarks merit deeper exploration.

show abstract