Teaching Small Language Models to Reason

Charlotte, Magister, Lucie; Mallinson, Jonathan; Adámek, Jakub; Malmi, Eric; Severyn, Aliaksei

doi:10.48550/arxiv.2212.08410

Cited by 14 publications

(24 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work exploits generating two types of teaching data from LLMs and mixing teaching data. We discover that this simple method highly improves student performance in complex multi-modality tasks, which has not yet been recognized in previous studies on fine-tuning with CoT reasoning (Hsieh et al 2023;Ho, Schmid, and Yun 2022;Huang et al 2022;Magister et al 2022;Fu et al 2023;Hu et al 2023).…”

Section: Related Workmentioning

confidence: 54%

See 1 more Smart Citation

T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Large Language Model Signals for Science Question Answering

Wang,

Hu,

et al. 2024

AAAI

View full text Add to dashboard Cite

Large Language Models (LLMs) have recently demonstrated exceptional performance in various Natural Language Processing (NLP) tasks. They have also shown the ability to perform chain-of-thought (CoT) reasoning to solve complex problems. Recent studies have explored CoT reasoning in complex multimodal scenarios, such as the science question answering task, by fine-tuning multimodal models with high-quality human-annotated CoT rationales. However, collecting high-quality COT rationales is usually time-consuming and costly. Besides, the annotated rationales are hardly accurate due to the external essential information missed. To address these issues, we propose a novel method termed T-SciQ that aims at teaching science question answering with LLM signals. The T-SciQ approach generates high-quality CoT rationales as teaching signals and is advanced to train much smaller models to perform CoT reasoning in complex modalities. Additionally, we introduce a novel data mixing strategy to produce more effective teaching data samples for simple and complex science question answer problems. Extensive experimental results show that our T-SciQ method achieves a new state-of-the-art performance on the ScienceQA benchmark, with an accuracy of 96.18%. Moreover, our approach outperforms the most powerful fine-tuned baseline by 4.5%. The code is publicly available at https://github.com/T-SciQ/T-SciQ.

show abstract

Section: Related Workmentioning

confidence: 54%

“…In recent studies, CoT reasoning is elicited in small models using fine-tuned language models. Magister et al (2022) benefit smaller models through CoT distillation. Huang et al (2022) show that LLMs can enhance reasoning using self-generated solutions from unlabeled data.…”

Section: Related Workmentioning

confidence: 99%

T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Large Language Model Signals for Science Question Answering

Wang,

Hu,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Some recent works also tackle the problem by distilling LLMs into smaller LMs for domain-specific tasks (Magister et al 2022;Hsieh et al 2023;Wang et al 2022a. Wang et al (2022a) propose PINTO, which uses LLMs' rationales as context of a small LM to improve its performance on reasoning tasks.…”

Section: Distilling Llmsmentioning

confidence: 99%

“…further propose SCOTT, a faithful knowledge distillation method that elicits self-consistent rationales from LLMs to fine-tune a small while more faithful LM for open-domain questionanswering tasks. Both Magister et al (2022) and Hsieh et al (2023) demonstrate that LLMs can be distilled into smaller but more effective LMs by fine-tuning with both answers and rationales on commonsense reasoning and arithmetic tasks. Different from these works, our work focus on the domain of toxic content detection.…”

Section: Distilling Llmsmentioning

confidence: 99%

Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Models

Zhang,

Wu,

et al. 2024

AAAI

View full text Add to dashboard Cite

Toxic content detection is crucial for online services to remove inappropriate content that violates community standards. To automate the detection process, prior works have proposed varieties of machine learning (ML) approaches to train Language Models (LMs) for toxic content detection. However, both their accuracy and transferability across datasets are limited. Recently, Large Language Models (LLMs) have shown promise in toxic content detection due to their superior zero-shot and few-shot in-context learning ability as well as broad transferability on ML tasks. However, efficiently designing prompts for LLMs remains challenging. Moreover, the high run-time cost of LLMs may hinder their deployments in production. To address these challenges, in this work, we propose BD-LLM, a novel and efficient approach to bootstrapping and distilling LLMs for toxic content detection. Specifically, we design a novel prompting method named Decision-Tree-of-Thought (DToT) to bootstrap LLMs' detection performance and extract high-quality rationales. DToT can automatically select more fine-grained context to re-prompt LLMs when their responses lack confidence. Additionally, we use the rationales extracted via DToT to fine-tune student LMs. Our experimental results on various datasets demonstrate that DToT can improve the accuracy of LLMs by up to 4.6%. Furthermore, student LMs fine-tuned with rationales extracted via DToT outperform baselines on all datasets with up to 16.9% accuracy improvement, while being more than 60x smaller than conventional LLMs. Finally, we observe that student LMs fine-tuned with rationales exhibit better cross-dataset transferability.

show abstract

“…This opens up new avenues of research, referred to as prompt engineering and meta-learning. Recent studies [11], [13], [18]- [20] highlight the significance and relevance of exploring these areas.…”

Section: Related Workmentioning

confidence: 99%

OntoChatGPT Information System: Ontology-Driven Structured Prompts for ChatGPT Meta-Learning

Палагин¹,

Kaverinskiy²,

Litvin³

et al. 2023

IJC

View full text Add to dashboard Cite

This research presents a comprehensive methodology for utilizing an ontology-driven structured prompts system in interplay with ChatGPT, a widely used large language model (LLM). The study develops formal models, both information and functional, and establishes the methodological foundations for integrating ontology-driven prompts with ChatGPT’s meta-learning capabilities. The resulting productive triad comprises the methodological foundations, advanced information technology, and the OntoChatGPT system, which collectively enhance the effectiveness and performance of chatbot systems. The implementation of this technology is demonstrated using the Ukrainian language within the domain of rehabilitation. By applying the proposed methodology, the OntoChatGPT system effectively extracts entities from contexts, classifies them, and generates relevant responses. The study highlights the versatility of the methodology, emphasizing its applicability not only to ChatGPT but also to other chatbot systems based on LLMs, such as Google’s Bard utilizing the PaLM 2 LLM. The underlying principles of meta-learning, structured prompts, and ontology-driven information retrieval form the core of the proposed methodology, enabling their adaptation and utilization in various LLM-based systems. This versatile approach opens up new possibilities for NLP and dialogue systems, empowering developers to enhance the performance and functionality of chatbot systems across different domains and languages.

show abstract

Teaching Small Language Models to Reason

Cited by 14 publications

References 0 publications

T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Large Language Model Signals for Science Question Answering

T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Large Language Model Signals for Science Question Answering

Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Models

OntoChatGPT Information System: Ontology-Driven Structured Prompts for ChatGPT Meta-Learning

Contact Info

Product

Resources

About