The Classification of Short Scientific Texts Using Pretrained BERT Model

Szostek, Małgorzata; Ishankulov, Timur; Kotik, Konstantin; Orlov, Yuriy; Shifrin, Michael; Потапов, А. А.

doi:10.3233/shti210125

Cited by 10 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Text classification can streamline the organization of large literature volumes by assigning predefined categories or labels to documents. This approach has been effectively applied to enhance reference prioritization in systematic reviews [17,18,19,20,21]. Text classification has also been used to better understand the structure of papers, for example by automatically predicting sections and headers in Electronic Health Records [22].…”

Section: Nlp Tasksmentioning

confidence: 99%

Large Language Models to process, analyze, and synthesize biomedical texts – a scoping review

Doneva,

Qin,

Sick

et al. 2024

Preprint

View full text Add to dashboard Cite

The advent of large language models (LLMs) such as BERT and, more recently, GPT, is transforming our approach of analyzing and understanding biomedical texts. To stay informed about the latest advancements in this area, there is a need for up-to-date summaries on the impact of LLM on Natural Language Processing (NLP) in the biomedical field. Thus, this scoping review aims to provide a detailed overview of the current state of biomedical NLP research and its applications, with a special focus on the evolving role of LLMs. We conducted a systematic search of PubMed, EMBASE, and Google Scholar for studies and conference proceedings published from 2017 to December 19, 2023, that develop or utilize LLMs for NLP tasks in biomedicine. From 13,823 references, we selected 199 publications and conference proceedings for our review. LLMs are being applied to a wide array of tasks in the biomedical field, including knowledge management, text mining, drug discovery, and evidence synthesis. Prominent among these tasks are text classification, relation extraction, and named entity recognition. Although BERT-based models remain prevalent, the use of GPT-based models has substantially increased since 2023.

show abstract

Section: Nlp Tasksmentioning

confidence: 99%

Large Language Models to process, analyze, and synthesize biomedical texts – a scoping review

Doneva,

Qin,

Sick

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The development of these methods is supported by the relevance language that models such as BERT [22] and GPT [23] are gaining. These pretrained models are getting more involved in natural language tasks like text classification [24], text summarization [25], [26], and text retrieval [27]. Embedding-based approach in extracting keywords could be seen in methods such as SIFRank [12], KBIR, and KeyBART [13] which involve embeddings from pre-trained language models, such as ELMo [28], RoBERTa [29], and BART [30] respectively, in determining the key terms to a document.…”

Section: B Automatic Keyword Extraction (Ake)mentioning

confidence: 99%

Expert Profile Identification From Community Detection on Author-Publication-Keyword Graph With Keyword Extraction

Fu,

Akbar

2024

IEEE Access

View full text Add to dashboard Cite

Expert profiling aims to discover the expertise of an author. This task is useful for identifying the research groups existing within an organization as well as measuring the similarities between authors' expertise. Thus, identifying areas of expertise becomes a critical part of this task, especially in cases where the publications given is unannotated. Commonly used topic modeling methods like Latent Dirichlet Allocation (LDA) still falls short when determining the number of topics automatically and discovering the hierarchical relationships between topics. To solve these issues, we adapted a graph-based approach which constructs a heterogeneous graph from publication features such as authors and keywords [1]. We applied Louvain algorithm repeatedly to automatically discover the areas of expertise in hierarchical order. From dataset exploration, we noticed a portion of entries in our dataset still lacks keywords, so we utilize keyword extraction methods to generate keywords from the title and abstract of each respective publication. We carried out experiment to determine the extraction methods, keyword combinations, and graph transformation weights with the optimum HPMI value. Results showed that graphs constructed from default and SIFRank keywords with transformation weights of α=0.5, β=1.0 produces topics with the best HPMI score. We also evaluate the quality of profiles from this graph-based model (CDT) and ATM as baseline. It is shown that CDT results in better MAP, MRR, and nDCG scores than ATM. The work in this manuscript shows how community detection and keyword extraction could be utilized in expert profiling. Our observation shows that the Louvain algorithm used only cluster publications into a topic only, and thus still has limitations in classifying multidisciplinary publications. Further development could be done to improve the keywords quality and handle such publications.

show abstract

“…Text mining plays a fundamental role in research [47,48] and medicine [49][50][51][52][53][54], and has important applications in several other fields, such as psychiatry [55], risk management [56], financial domains [57], finance [58], service management [59], social networks [60], social media [61], education and training [62], policy-making [63], and agriculture [64], among others.…”

Section: Text Miningmentioning

confidence: 99%

Predictive Maintenance with Linguistic Text Mining

Postiglione,

Monteleone

2024

Mathematics

View full text Add to dashboard Cite

The escalating intricacy of industrial systems necessitates strategies for augmenting the reliability and efficiency of industrial machinery to curtail downtime. In such a context, predictive maintenance (PdM) has surfaced as a pivotal strategy. The amalgamation of cyber-physical systems, IoT devices, and real-time data analytics, emblematic of Industry 4.0, proffers novel avenues to refine maintenance of production equipment from both technical and managerial standpoints, serving as a supportive technology to enhance the precision and efficacy of predictive maintenance. This paper presents an innovative approach that melds text mining techniques with the cyber-physical infrastructure of a manufacturing sector. The aim is to improve the precision and promptness of predictive maintenance within industrial settings. The text mining framework is designed to sift through extensive log files containing data on the status of operational parameters. These datasets encompass information generated by sensors or computed by the control system throughout the production process execution. The algorithm aids in forecasting potential equipment failures, thereby curtailing maintenance costs and fortifying overall system resilience. Furthermore, we substantiate the efficacy of our approach through a case study involving a real-world industrial machine. This research contributes to the progression of predictive maintenance strategies by leveraging the wealth of textual information available within industrial environments, ultimately bolstering equipment reliability and operational efficiency.

show abstract

The Classification of Short Scientific Texts Using Pretrained BERT Model

Cited by 10 publications

References 7 publications

Large Language Models to process, analyze, and synthesize biomedical texts – a scoping review

Large Language Models to process, analyze, and synthesize biomedical texts – a scoping review

Expert Profile Identification From Community Detection on Author-Publication-Keyword Graph With Keyword Extraction

Predictive Maintenance with Linguistic Text Mining

Contact Info

Product

Resources

About