Real-time style transfer with efficient vision transformers

Benmeziane, Hadjer; Ouarnoughi, Hamza; Maghraoui, Kaoutar El; Niar, Smaïl

doi:10.1145/3517206.3526271

Cited by 4 publications

(1 citation statement)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…. REAL-TIME STYLE TRANSFERBenmeziane et al performed Real-time Style Transfer[224] on Xiaomi Redmi 10 mobile and Raspberry Pi 3 embedded platform by searching efficient ViTs. Style Transfer[225],[226] is a vision task of transferring the style of an image to the content of a different image.…”

mentioning

confidence: 99%

Neural Architecture Search for Transformers: A Survey

et al. 2022

View full text Add to dashboard Cite

Transformer-based Deep Neural Network architectures have gained tremendous interest due to their effectiveness in various applications across Natural Language Processing (NLP) and Computer Vision (CV) domains. These models are the de facto choice in several language tasks, such as Sentiment Analysis and Text Summarization, replacing Long Short Term Memory (LSTM) model. Vision Transformers (ViTs) have shown better model performance than traditional Convolutional Neural Networks (CNNs) in vision applications while requiring significantly fewer parameters and training time. The design pipeline of a neural architecture for a given task and dataset is extremely challenging as it requires expertise in several interdisciplinary areas such as signal processing, image processing, optimization and allied fields. Neural Architecture Search (NAS) is a promising technique to automate the architectural design process of a Neural Network in a data-driven way using Machine Learning (ML) methods. The search method explores several architectures without requiring significant human effort, and the searched models outperform the manually built networks. In this paper, we review Neural Architecture Search techniques, targeting the Transformer model and its family of architectures such as Bidirectional Encoder Representations from Transformers (BERT) and Vision Transformers. We provide an in-depth literature review of approximately 50 state-of-theart Neural Architecture Search methods and explore future directions in this fast-evolving class of problems.

show abstract

mentioning

confidence: 99%