ChatGPT has Aced the Test of Understanding in College Economics: Now What?

Geerling, Wayne; Mateer, G. Dirk; Wooten, Jadrian; Damodaran, Nikhil

doi:10.1177/05694345231169654

Cited by 50 publications

(37 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They were also found to be struggling with tasks that require quantitative reasoning, such as solving mathematics, science, and engineering problems [6,7], and were likely to make up false facts [8]. Since the release of ChatGPT, this field of research has significantly expanded, with scholars in different fields studying ChatGPT's performance in their subject areas [9][10][11][12]. Scholars generally find the progress in the development of the technology impressive, but also point out its limited abilities in different areas.…”

Section: Existing Research On Llm Performance On Physics Tasksmentioning

confidence: 99%

“…In computational linguistics, this ability is often referred to as the ability to 'reason' despite this obviously being a misnomer. 10 In computational linguistics, anthropomorphising terms like 'reasoning' and 'thought' are often used to refer to the abilities of LLMs, which we would typically ascribe to people displaying similar behaviours. In the remainder of this section, we also use the terms 'reasoning' and 'thought' in a way that reflects their use in the existing literature on LLMs.…”

Section: The Chain-of-thought Approachesmentioning

confidence: 99%

“…A common approach to ICL is to begin the prompt by providing one or more examples of tasks accompanied by solutions, to create a demonstration context, and then providing the task to be answered by the LLM. In this way, the LLM can learn from analogy directly from 10 Granted, the computational linguistic community is likely less concerned with the epistemological considerations of physics learning than the PER community. 11 Cognitive linguists have compared this step-by-step approach to solving problems to 'thinking slow', by analogy to the dual-process theories of human cognition [65,66].…”

Section: 31mentioning

confidence: 99%

See 2 more Smart Citations

How understanding large language models can inform the use of ChatGPT in physics education

Polverini,

Gregorcic

2024

Eur. J. Phys.

View full text Add to dashboard Cite

The paper aims to fulfil three main functions: (1) to serve as an introduction for the physics education community to the functioning of Large Language Models (LLMs), (2) to present a series of illustrative examples demonstrating how prompt-engineering techniques can impact LLMs performance on conceptual physics tasks and (3) to discuss potential implications of the understanding of LLMs and prompt engineering for physics teaching and learning. We first summarise existing research on the performance of a popular LLM-based chatbot (ChatGPT) on physics tasks. We then give a basic account of how LLMs work, illustrate essential features of their functioning, and discuss their strengths and limitations. Equipped with this knowledge, we discuss some challenges with generating useful output with ChatGPT-4 in the context of introductory physics, paying special attention to conceptual questions and problems. We then provide a condensed overview of relevant literature on prompt engineering and demonstrate through illustrative examples how selected prompt-engineering techniques can be employed to improve ChatGPT-4’s output on conceptual introductory physics problems. Qualitatively studying these examples provides additional insights into ChatGPT’s functioning and its utility in physics problem solving. Finally, we consider how insights from the paper can inform the use of LMMs in the teaching and learning of physics.

show abstract

Section: Existing Research On Llm Performance On Physics Tasksmentioning

confidence: 99%

Section: The Chain-of-thought Approachesmentioning

confidence: 99%

Section: 31mentioning

confidence: 99%

See 1 more Smart Citation

How understanding large language models can inform the use of ChatGPT in physics education

Polverini,

Gregorcic

2024

Eur. J. Phys.

View full text Add to dashboard Cite

show abstract

“…Academics, students, and researchers will use LLM tools (Geerling et al, 2023). We argue that fact-checking is necessary to prevent false statements from propagating in the literature.…”

Section: Discussionmentioning

confidence: 99%

ChatGPT Hallucinates Non-existent Citations: Evidence from Economics

Buchanan,

Hill,

Shapoval

2023

The American Economist

View full text Add to dashboard Cite

In this study, we generate prompts derived from every topic within the Journal of Economic Literature to assess the abilities of both GPT-3.5 and GPT-4 versions of the ChatGPT large language model (LLM) to write about economic concepts. ChatGPT demonstrates considerable competency in offering general summaries but also cites non-existent references. More than 30% of the citations provided by the GPT-3.5 version do not exist and this rate is only slightly reduced for the GPT-4 version. Additionally, our findings suggest that the reliability of the model decreases as the prompts become more specific. We provide quantitative evidence for errors in ChatGPT output to demonstrate the importance of LLM verification.JEL Codes: B4; O33; I2

show abstract

“…Al Ghatrifi et al (2023) [75], Geerling (2023) [76], Skavronskaya et al (2023) [77], Elnozahy and El Khayat (2023) [78], Aydın and Ayhan Erdem (2022) [79], Cooper (2023) [80] Ye and Bors (2022) [93].…”

Section: Acknowledgmentsmentioning

confidence: 99%

Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis

Bahroun,

Anane,

Ahmed

et al. 2023

Sustainability

181

View full text Add to dashboard Cite

In the ever-evolving era of technological advancements, generative artificial intelligence (GAI) emerges as a transformative force, revolutionizing education. This review paper, guided by the PRISMA framework, presents a comprehensive analysis of GAI in education, synthesizing key insights from a selection of 207 research papers to identify research gaps and future directions in the field. This study begins with a content analysis that explores GAI’s transformative impact in specific educational domains, including medical education and engineering education. The versatile applications of GAI encompass assessment, personalized learning support, and intelligent tutoring systems. Ethical considerations, interdisciplinary collaboration, and responsible technology use are highlighted, emphasizing the need for transparent GAI models and addressing biases. Subsequently, a bibliometric analysis of GAI in education is conducted, examining prominent AI tools, research focus, geographic distribution, and interdisciplinary collaboration. ChatGPT emerges as a dominant GAI tool, and the analysis reveals significant and exponential growth in GAI research in 2023. Moreover, this paper identifies promising future research directions, such as GAI-enhanced curriculum design and longitudinal studies tracking its long-term impact on learning outcomes. These findings provide a comprehensive understanding of GAI’s potential in reshaping education and offer valuable insights to researchers, educators, and policymakers interested in the intersection of GAI and education.

show abstract

ChatGPT has Aced the Test of Understanding in College Economics: Now What?

Cited by 50 publications

References 36 publications

How understanding large language models can inform the use of ChatGPT in physics education

How understanding large language models can inform the use of ChatGPT in physics education

ChatGPT Hallucinates Non-existent Citations: Evidence from Economics

Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis

Contact Info

Product

Resources

About