Generating Diverse Code Explanations using the GPT-3 Large Language Model

MacNeil, Stephen; Tran, Andrew; Mogil, Dan; Bernstein, Seth; Ross, E. Clarke; Huang, Ziheng

doi:10.1145/3501709.3544280

Cited by 120 publications

(48 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Language Model: A language model is a type of artificial intelligence model is trained to generate text that is similar to human language (MacNeil et al, 2022).…”

Section: Some Key Concepts Related To Chatgptmentioning

confidence: 99%

Chatting about ChatGPT: how may AI and GPT impact academia and libraries?

Lund

Wang²

2023

LHTN

641

147

View full text Add to dashboard Cite

Purpose This paper aims to provide an overview of key definitions related to ChatGPT, a public tool developed by OpenAI, and its underlying technology, Generative Pretrained Transformer (GPT). Design/methodology/approach This paper includes an interview with ChatGPT on its potential impact on academia and libraries. The interview discusses the benefits of ChatGPT such as improving search and discovery, reference and information services; cataloging and metadata generation; and content creation, as well as the ethical considerations that need to be taken into account, such as privacy and bias. Findings ChatGPT has considerable power to advance academia and librarianship in both anxiety-provoking and exciting new ways. However, it is important to consider how to use this technology responsibly and ethically, and to uncover how we, as professionals, can work alongside this technology to improve our work, rather than to abuse it or allow it to abuse us in the race to create new scholarly knowledge and educate future professionals. Originality/value This paper discusses the history and technology of GPT, including its generative pretrained transformer model, its ability to perform a wide range of language-based tasks and how ChatGPT uses this technology to function as a sophisticated chatbot.

show abstract

“…Language Model: A language model is a type of artificial intelligence model is trained to generate text that is similar to human language (MacNeil et al, 2022).…”

Section: Some Key Concepts Related To Chatgptmentioning

confidence: 99%

Chatting about ChatGPT: how may AI and GPT impact academia and libraries?

Lund

Wang²

2023

LHTN

641

147

View full text Add to dashboard Cite

show abstract

“…In computing education, a recent work by MacNeil et al [19] has employed GPT-3 to generate code explanations. Despite several open research and pedagogical questions that need to be further explored, this work has successfully demonstrated the potential of GPT-3 to support learning by explaining aspects of a given code snippet.…”

Section: Review Of Research Applying Large Languagementioning

confidence: 99%

ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education

Kasneci¹,

Seßler²,

Küchemann³

et al. 2023

Preprint

133

View full text Add to dashboard Cite

Large language models represent a significant advancement in the field of AI. The underlying technology is key to further innovations and, despite critical views and even bans within communities and regions, large language models are here to stay. This position paper presents the potential benefits and challenges of educational applications of large language models, from student and teacher perspectives. We briefly discuss the current state of large language models and their applications. We then highlight how these models can be used to create educational content, improve student engagement and interaction, and personalize learning experiences. With regard to challenges, we argue that large language models in education require teachers and learners to develop sets of competencies and literacies necessary to both understand the technology as well as their limitations and unexpected brittleness of such systems. In addition, a clear strategy within educational systems and a clear pedagogical approach with a strong focus on critical thinking and strategies for fact checking are required to integrate and take full advantage of large language models in learning settings and teaching curricula. Other challenges such as the potential bias in the output, the need for continuous human oversight, and the potential for misuse are not unique to the application of AI in education. But we believe that, if handled sensibly, these challenges can offer insights and opportunities in education scenarios to acquaint students early on with potential societal biases, criticalities, and risks of AI applications. We conclude with recommendations for how to address these challenges and ensure that such models are used in a responsible and ethical manner in education.

show abstract

“…This was sufficient for our study: an explanation failure occurred only 13 times out of 159 queries in the grounded condition, on average 1.08 times per user. We do not claim that our algorithm is the most effective, and future work could explore alternative ways of producing grounded utterances, such as leveraging the LLM itself [45,72]. Our system is limited in assuming a single well-defined relational data table.…”

Section: Limitationsmentioning

confidence: 97%

“What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models

Liu¹,

Sarkar²,

Negreanu³

et al. 2023

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Code-generating large language models map natural language to code. However, only a small portion of the infinite space of naturalistic utterances is effective at guiding code generation. For non-expert end-user programmers, learning this is the challenge of abstraction matching. We examine this challenge in the specific context of data analysis in spreadsheets, in a system that maps the user's natural language query to Python code using the Codex generator, executes the code, and shows the result. We propose grounded abstraction matching, which bridges the abstraction gap by translating the code back into a systematic and predictable naturalistic utterance. In a between-subjects, think-aloud study (n=24), we compare grounded abstraction matching to an ungrounded alternative based on previously established query framing principles. We find that the grounded approach improves end-users' understanding of the scope and capabilities of the code-generating model, and the kind of language needed to use it effectively. CCS CONCEPTS• Human-centered computing → Natural language interfaces; Interactive systems and tools; Empirical studies in HCI .

show abstract

Generating Diverse Code Explanations using the GPT-3 Large Language Model

Cited by 120 publications

References 13 publications

Chatting about ChatGPT: how may AI and GPT impact academia and libraries?

Chatting about ChatGPT: how may AI and GPT impact academia and libraries?

ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education

“What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models

Contact Info

Product

Resources

About