A Survey on BERT and Its Applications

Aftan, Sulaiman; Shah, Habib

doi:10.1109/lt58159.2023.10092289

Cited by 13 publications

(1 citation statement)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BERT [70][71][72][73][74][75][76], introduced by Google researchers in 2018 [70], has significantly advanced language-related tasks and was integrated into Google's search engine by 2019. An analysis published in 2020 highlighted BERT's rapid ascent in the research world, with over 150 scientific publications within just over a year [71].…”

Section: Bert (Bidirectional Encoder Representations From Transformers)mentioning

confidence: 99%

Finite State Automata on Multi-Word Units for Efficient Text-Mining

Postiglione

2024

Mathematics

View full text Add to dashboard Cite

Text mining is crucial for analyzing unstructured and semi-structured textual documents. This paper introduces a fast and precise text mining method based on a finite automaton to extract knowledge domains. Unlike simple words, multi-word units (such as credit card) are emphasized for their efficiency in identifying specific semantic areas due to their predominantly monosemic nature, their limited number and their distinctiveness. The method focuses on identifying multi-word units within terminological ontologies, where each multi-word unit is associated with a sub-domain of ontology knowledge. The algorithm, designed to handle the challenges posed by very long multi-word units composed of a variable number of simple words, integrates user-selected ontologies into a single finite automaton during a fast pre-processing step. At runtime, the automaton reads input text character by character, efficiently locating multi-word units even if they overlap. This approach is efficient for both short and long documents, requiring no prior training. Ontologies can be updated without additional computational costs. An early system prototype, tested on 100 short and medium-length documents, recognized the knowledge domains for the vast majority of texts (over 90%) analyzed. The authors suggest that this method could be a valuable semantic-based knowledge domain extraction technique in unstructured documents.

show abstract

Section: Bert (Bidirectional Encoder Representations From Transformers)mentioning

confidence: 99%

Finite State Automata on Multi-Word Units for Efficient Text-Mining

Postiglione

2024

Mathematics

View full text Add to dashboard Cite

show abstract

Arabic News Articles Classification Using Different Word Embeddings

Khaled,

Al-Barham,

Alomari

et al. 2024

Lecture Notes in Networks and Systems

View full text Add to dashboard Cite

A comprehensive review of model compression techniques in machine learning

Dantas,

Sabino da Silva,

Cordeiro

et al. 2024

Appl Intell

View full text Add to dashboard Cite

This paper critically examines model compression techniques within the machine learning (ML) domain, emphasizing their role in enhancing model efficiency for deployment in resource-constrained environments, such as mobile devices, edge computing, and Internet of Things (IoT) systems. By systematically exploring compression techniques and lightweight design architectures, it is provided a comprehensive understanding of their operational contexts and effectiveness. The synthesis of these strategies reveals a dynamic interplay between model performance and computational demand, highlighting the balance required for optimal application. As machine learning (ML) models grow increasingly complex and data-intensive, the demand for computational resources and memory has surged accordingly. This escalation presents significant challenges for the deployment of artificial intelligence (AI) systems in real-world applications, particularly where hardware capabilities are limited. Therefore, model compression techniques are not merely advantageous but essential for ensuring that these models can be utilized across various domains, maintaining high performance without prohibitive resource requirements. Furthermore, this review underscores the importance of model compression in sustainable artificial intelligence (AI) development. The introduction of hybrid methods, which combine multiple compression techniques, promises to deliver superior performance and efficiency. Additionally, the development of intelligent frameworks capable of selecting the most appropriate compression strategy based on specific application needs is crucial for advancing the field. The practical examples and engineering applications discussed demonstrate the real-world impact of these techniques. By optimizing the balance between model complexity and computational efficiency, model compression ensures that the advancements in AI technology remain sustainable and widely applicable. This comprehensive review thus contributes to the academic discourse and guides innovative solutions for efficient and responsible machine learning practices, paving the way for future advancements in the field. Graphical abstract

show abstract

A Survey on BERT and Its Applications

Cited by 13 publications

References 26 publications

Finite State Automata on Multi-Word Units for Efficient Text-Mining

Finite State Automata on Multi-Word Units for Efficient Text-Mining

Arabic News Articles Classification Using Different Word Embeddings

A comprehensive review of model compression techniques in machine learning

Contact Info

Product

Resources

About