Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks

Peris, Charith; Tan, Lizhen; Gueudré, Thomas; Gojayev, Turan; Pan, Wei; Oz, Gokmen

doi:10.18653/v1/2022.emnlp-industry.12

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track 2022

DOI: 10.18653/v1/2022.emnlp-industry.12

|View full text |Cite

Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks

Charith Peris¹,

Lizhen Tan²,

Thomas Gueudré³

et al.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Other1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Inference Optimization of Foundation Models on AI Accelerators

Park,

Budhathoki,

Chen

et al. 2024

Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Powerful foundation models, including large language models (LLMs), with Transformer architectures have ushered in a new era of Generative AI across various industries. Industry and research community have witnessed a large number of new applications, based on those foundation models. Such applications include question and answer, customer services, image and video generation, and code completions, among others. However, as the number of model parameters reaches to hundreds of billions, their deployment incurs prohibitive inference costs and high latency in real-world scenarios. As a result, the demand for cost-effective and fast inference using AI accelerators is ever more higher. To this end, our tutorial offers a comprehensive discussion on complementary inference optimization techniques using AI accelerators. Beginning with an overview of basic Transformer architectures and deep learning system frameworks, we deep dive into system optimization techniques for fast and memory-efficient attention computations and discuss how they can be implemented efficiently on AI accelerators. Next, we describe architectural elements that are key for fast transformer inference. Finally, we examine various model compression and fast decoding strategies in the same context.

show abstract

Inference Optimization of Foundation Models on AI Accelerators

Park,

Budhathoki,

Chen

et al. 2024

Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Knowledge Distillation Transfer Sets and their Impact on Downstream NLU Tasks

Cited by 1 publication

References 0 publications

Inference Optimization of Foundation Models on AI Accelerators

Inference Optimization of Foundation Models on AI Accelerators

Contact Info

Product

Resources

About