Automated Knowledge Organization AI ML based Subject Indexing System for Libraries

Ahmed, Moamin; Mukhopadhyay, Mondrita; Mukhopadhyay, Parthasarathi

doi:10.14429/djlit.43.01.18619

Cited by 5 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This The overview diagram of our data processing and computational framework starting from data and ending with visualizations. tool was chosen mainly for the following reasons: (1) it's proof-tested for quality and robustness by previous works (see, e.g., Suominen et al, 2022;Ahmed et al, 2023;Inkinen et al, 2023), (2) it's used by various public libraries in Finland, Germany and Sweden, ( 3) its opensource and trainable with own data, (4) it comes pretrained for Finnish, English and Swedish, and ( 5) it uses generic YSO ontology suitable for all types of texts. Annif 's framework comprises a lexical subject indexing algorithm for finding correlations between subjects in the vocabulary and words in documents, a text classification algorithm, and a general-purpose machine learning algorithm.…”

Section: Computation Of Topicsmentioning

confidence: 99%

Topic mining for theses and job ads in ICT sector: can higher education institutes respond to job market demands?

Kauttonen,

Khan,

Aunimo

et al. 2024

Front. Educ.

View full text Add to dashboard Cite

IntroductionThis study aims to tackle the challenge of ensuring higher education students are equipped with high-demand skills for today’s job market. The focus is on aligning the knowledge acquired during their studies, as represented by final-year thesis projects, with the skills and topics specified in actual job advertisements.MethodsWe developed a computational framework that uses automated subject indexing to extract representative skills and topics from two major datasets: thesis abstracts from Information and Communication Technology (ICT) programmes of Finnish Universities of Applied Sciences, and ICT-related job ads from a top Finnish job portal. Our dataset spans 12 years, comprising 18,254 theses and 107,335 ads. The framework includes a subject indexing model for keyword extraction, dimension reduction techniques for data simplification, clustering algorithms to group similar items, and correlation analysis to compare similarities and differences between the two datasets.ResultsThe analysis uncovered both similarities and differences between thesis topics and trends in job ads. It highlighted areas where education aligns with industry demands but also pointed out existing gaps.DiscussionOur framework not only helps to align the education provided with industry demands but also ensures that higher education institutes can stay up-to-date with the latest skills and knowledge in the field, thereby better equipping students for success in their careers. While the framework was applied to the ICT sector in this instance, its design allows expansion into other fields offering a data-informed approach for continuous development of teaching curricula and methodologies.

show abstract

Section: Computation Of Topicsmentioning

confidence: 99%

Topic mining for theses and job ads in ICT sector: can higher education institutes respond to job market demands?

Kauttonen,

Khan,

Aunimo

et al. 2024

Front. Educ.

View full text Add to dashboard Cite

show abstract

Automatic Indexing for Agriculture: Designing a Framework by Deploying Agrovoc, Agris and Annif

Ahmed

2023

srels

View full text Add to dashboard Cite

There are several ways to employ machine learning for automating subject indexing. One popular strategy is to utilize a supervised learning algorithm to train a model on a set of documents that have been manually indexed by subject matter using a standard vocabulary. The resulting model can then predict the subject of new and previously unseen documents by identifying patterns learned from the training data. To do this, the first step is to gather a large dataset of documents and manually assign each document a set of subject keywords/descriptors from a controlled vocabulary (e.g., from Agrovoc). Next, the dataset (obtained from Agris) can be divided into – i) a training dataset, and ii) a test dataset. The training dataset is used to train the model, while the test dataset is used to evaluate the model's performance. Machine learning can be a powerful tool for automating the process of subject indexing. This research is an attempt to apply Annif (http://annif. org/), an open-source AI/ML framework, to autogenerate subject keywords/descriptors for documentary resources in the domain of agriculture. The training dataset is obtained from Agris, which applies the Agrovoc thesaurus as a vocabulary tool (https://www.fao.org/agris/download).

show abstract

Categorisation of Indian Research Publications by Sustainable Development Goals (SDGs): A Machine Learning Approach

Pal,

Mukhopadhyay

2024

srels

View full text Add to dashboard Cite

This research explores the feasibility of automatically clustering and categorising Indian research output in terms of the seventeen categories of Sustainable Development Goals (SDGs) proposed by the United Nations (sdgs.un.org/goals). Utilising the OpenAlex database (openalex.org), an extensive open-access bibliographic repository containing over 250 million research objects, this study gathered research publications from India (with at least one Indian author) published during 2016 (the year of inception of the SDG categories) - 2023. OpenAlex’s comprehensive metadata includes SDG data elements for numerous publications, alongside other bibliographic information. The SDG classifications and corresponding accuracy scores (ranging from 0 to 1) in OpenAlex are derived from the Aurora SDG classifier (aurora-universities. eu/ sdg-research/). Initially, this study compiled a dataset of 500,000 research publications originating from India within the specified period, of which approximately 278,845 records contained SDG data elements. This primary dataset was divided into two subsets: the All Dataset (ADS), encompassing all records (278,845 records), and the High Accuracy Dataset (HAD), consisting of records with an SDG accuracy score of ≥0.5 (204,793 records). Both datasets (ADS and HAD) were further segmented into three groups: the training dataset (97% of total data), the validation dataset (1.5% of total data), and the test dataset (1.5% of total data). The training dataset was utilised to train the locally implemented open-source automated indexing tool, Annif, using various backends, including lexical, associative, and ensemble (simple) methods. The validation dataset was employed to develop an optimal weightage formula for combining results from different backends in an advanced-level neural network backend. One significant advantage of the neural network backend is its capacity for successive training with additional datasets. It was found that: (i) automatic categorisation of research publications by SDGs is feasible; (ii) the neural network-based backend outperformed other backends, such as SVC, Omikuji, and FastText, in terms of retrieval metrics like F1@5 and NDCG; (iii) lexical models are unsuitable for this purpose, performing poorly in terms of F1@5 and NDCG; and (iv) the neural network-based backend also outperformed other backends for the High Accuracy Dataset.

show abstract

Automated Knowledge Organization AI ML based Subject Indexing System for Libraries

Cited by 5 publications

References 0 publications

Topic mining for theses and job ads in ICT sector: can higher education institutes respond to job market demands?

Topic mining for theses and job ads in ICT sector: can higher education institutes respond to job market demands?

Automatic Indexing for Agriculture: Designing a Framework by Deploying Agrovoc, Agris and Annif

Categorisation of Indian Research Publications by Sustainable Development Goals (SDGs): A Machine Learning Approach

Contact Info

Product

Resources

About