Where are the Large N Studies in Education?

Corlatescu, Dragos; Ruşeţi, Ştefan; Toma, Irina; Dascălu, Mihai

doi:10.1145/3491140.3528315

Cited by 2 publications

(10 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While focusing on the task of extracting large-scale studies, the authors of ref. [8] managed to achieve an F1 score of 0.633 on a manually annotated corpus of 1000 articles by using multiple heuristics based on advanced text searches, regular expressions, and Natural Language Processing (NLP) techniques such as speech tagging and syntactic dependency parsing. They also applied the same strategy to a corpus of around 33,531 full-text articles extracted by crawling several journals and conference websites such as Frontiers, CHB, CE, L@S, AIED, and EC-TEL.…”

Section: Extracting Relevant Information From Scientific Articlesmentioning

confidence: 99%

“…Our first corpus is used to find the most suitable prompts. This corpus comprises 20 articles selected from a group of 50 randomly extracted articles from the EdPub corpus introduced by [8]. The 20 articles were selected based on their differences.…”

Section: Datasetsmentioning

confidence: 99%

“…Second, a newly annotated corpus of 200 articles was used to validate the results. The articles were selected randomly from a larger corpus extracted from crawling multiple Internet sources [8]. The information available for each article consisted of the publication venue, the year when the paper was published, the title, whether it was a study that employed more than 1000 participants, and the total number of individuals involved in the studies.…”

Section: Datasetsmentioning

confidence: 99%

“…More specifically, we prompt an FLAN-T5 XXL language model [7] with targeted questions and paragraphs from a testing corpus such that the model eventually returns the number of participants from the study. We build on the work of Corlatescu et al [8] and introduce a generalizable approach in which we use large language models to answer specific questions about research papers.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Asking Questions about Scientific Articles—Identifying Large N Studies with LLMs

Paroiu,

Ruseti,

Dascalu

et al. 2023

Electronics

Self Cite

View full text Add to dashboard Cite

The exponential growth of scientific publications increases the effort required to identify relevant articles. Moreover, the scale of studies is a frequent barrier to research as the majority of studies are low or medium-scaled and do not generalize well while lacking statistical power. As such, we introduce an automated method that supports the identification of large-scale studies in terms of population. First, we introduce a training corpus of 1229 manually annotated paragraphs extracted from 20 articles with different structures and considered populations. Our method considers prompting a FLAN-T5 language model with targeted questions and paragraphs from the previous corpus so that the model returns the number of participants from the study. We adopt a dialogic extensible approach in which the model is asked a sequence of questions that are gradual in terms of focus. Second, we use a validation corpus with 200 articles labeled for having N larger than 1000 to assess the performance of our language model. Our model, without any preliminary filtering with heuristics, achieves an F1 score of 0.52, surpassing previous analyses performed that obtained an F1 score of 0.51. Moreover, we achieved an F1 score of 0.69 when combined with previous extraction heuristics, thus arguing for the robustness and extensibility of our approach. Finally, we apply our model to a newly introduced dataset of ERIC publications to observe trends across the years in the Education domain. A spike was observed in 2019, followed by a decrease in 2020 and, afterward, a positive trend; nevertheless, the overall percentage is lower than 3%, suggesting a major problem in terms of scale and the need for a change in perspective.

show abstract

Section: Extracting Relevant Information From Scientific Articlesmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Asking Questions about Scientific Articles—Identifying Large N Studies with LLMs

Paroiu,

Ruseti,

Dascalu

et al. 2023

Electronics

Self Cite

View full text Add to dashboard Cite

show abstract

“…The development of Large Language Models (LLMs) has significantly advanced the field of Natural Language Processing (NLP), enabling machines to comprehend and generate human language with unprecedented accuracy and fluency [1,2].…”

Section: Introductionmentioning

confidence: 99%

A study on factors affecting the decision of choosing university of high school pupils in Quang Ngai province

Nguyen¹

2021

VMOST

View full text Add to dashboard Cite

In fact, choosing a career and a university of high school pupils is very important. This study aims at identifying and measuring the factors affecting the decision of choosing a university of high school pupils in Quang Ngai province. The data of this study were collected from 340 answer sheets of grade 12 pupils of 5 high schools in Quang Ngai province in the 2019-2020 school year. By using the quantitative analysis methods, it has been shown that 5 factors are affecting the decision of choosing a university of pupils in Quang Ngai province, in the order of high importance to low including (1) University reputation, (2) Communication, (3) Learning conditions, (4) Factor belongs to the pupils, (5) Individuals.

show abstract

Where are the Large N Studies in Education?

Cited by 2 publications

References 3 publications

Asking Questions about Scientific Articles—Identifying Large N Studies with LLMs

Asking Questions about Scientific Articles—Identifying Large N Studies with LLMs

A study on factors affecting the decision of choosing university of high school pupils in Quang Ngai province

Contact Info

Product

Resources

About