Architecture to improve the accuracy of automatic image annotation systems

Khatchatoorian, Artin Ghostan; Jamzad, Mansour

doi:10.1049/iet-cvi.2019.0500

Cited by 10 publications

(4 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vatani From Table 2, it appears that our proposed segmentation-based AIA method outperforms the majority of the stated related works in both scenarios of 274 and 260 concepts If we take as an instance the top two F1 scores yielded by the related works Khatchatoorian et al (2020) [72] and CNN-THOP (2020) [74] in the scenario of 260 concepts, we can clearly see that the outcomes of our method exceed those of both methods by 5% at least. Furthermore, the F1 score obtained by our method is at least 10% higher than that obtained by other recent studies such as GCN (2020) [73], SSL-AWF (2021) [81], and MVRSC (2021) [82].…”

Section: Regiomentioning

confidence: 77%

“…By understanding the logic that connects different concepts, the system became able to learn concepts regardless of their narrow use. On the other hand, the idea in the work of Khatchatoorian et al (2020) [72] revolves around employing CNN as a black box and letting it learn everything by itself. However, we have taken advantage of both the methods by applying a CNN to obtain a rich set of features representing the concepts and employing KNN regression to understand how these concepts are related.…”

Section: Regiomentioning

confidence: 99%

See 1 more Smart Citation

Deep Convolutional Neural Network with KNN Regression for Automatic Image Annotation

et al. 2021

View full text Add to dashboard Cite

Automatic image annotation is an active field of research in which a set of annotations are automatically assigned to images based on their content. In literature, some works opted for handcrafted features and manual approaches of linking concepts to images, whereas some others involved convolutional neural networks (CNNs) as black boxes to solve the problem without external interference. In this work, we introduce a hybrid approach that combines the advantages of both CNN and the conventional concept-to-image assignment approaches. J-image segmentation (JSEG) is firstly used to segment the image into a set of homogeneous regions, then a CNN is employed to produce a rich feature descriptor per area, and then, vector of locally aggregated descriptors (VLAD) is applied to the extracted features to generate compact and unified descriptors. Thereafter, the not too deep clustering (N2D clustering) algorithm is performed to define local manifolds constituting the feature space, and finally, the semantic relatedness is calculated for both image–concept and concept–concept using KNN regression to better grasp the meaning of concepts and how they relate. Through a comprehensive experimental evaluation, our method has indicated a superiority over a wide range of recent related works by yielding F1 scores of 58.89% and 80.24% with the datasets Corel 5k and MSRC v2, respectively. Additionally, it demonstrated a relatively high capacity of learning more concepts with higher accuracy, which results in N+ of 212 and 22 with the datasets Corel 5k and MSRC v2, respectively.

show abstract

Section: Regiomentioning

confidence: 77%

Section: Regiomentioning

confidence: 99%

Deep Convolutional Neural Network with KNN Regression for Automatic Image Annotation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Upon examining the values of N + , it is evident that both Query2Label and the proposed network have predicted more than 10 labels not previously annotated by the GAP-based method, demonstrating their proficiency in annotating tags that occur less frequently. Except for certain approaches like [52] that have attempted to annotate low-frequency tags by selecting a distinct threshold for each one, extracting the image features associated with these tags using traditional CNNs is challenging. However, it is important to consider that determining the optimal threshold each label is impacted by the training data, which can constrain the model's generalizability.…”

Section: Comparisons Of Different Attention Modulesmentioning

confidence: 99%

Enhancing high-vocabulary image annotation with a novel attention-based pooling

Salar,

Ahmadi

2023

Preprint

View full text Add to dashboard Cite

Given an image, we aim to assign a set of semantic labels to its visual content automatically. This is generally known as automatic image annotation (AIA). Images contain objects that can vary in size and position, with some only taking up a small region of the entire picture. The rise in the number of object classes also heightens this variety. Despite the achievement of promising results, the majority of current methods have limited efficacy in the detection of small-scale objects. To make more effective use of spatial data compared to the global pooling method, we propose a modified transformer decoder layer that improves computational complexity without sacrificing model performance. The study has conducted multiple experiments on four datasets, including three high-vocabulary small-scale datasets (Corel 5k, IAPR TC-12, and Esp Game) and one large-scale dataset (Visual Genome) with a vocabulary list of 500 words. In comparison to existing state-of-the-art models, our approach achieves comparable results in F1-score, \({\text{N}}^{+}\), and mean average precision (mAP) on small- and large-scale datasets.

show abstract

“…The interconnection mechanisms are so basic (method invocation); thus, complex interconnections are so hard in these methods. In a nutshell, there is no clear image of system architecture before component creation [ 42 ]. Component-based SA relieves this modeling trouble and develops reusable off-the-shelf component-based heterogeneous systems.…”

Section: Software Architecture Designmentioning

confidence: 99%

Lightweight Software Architecture Evaluation for Industry: A Comprehensive Review

Sahlabadi

Muniyandi

Shukur

et al. 2022

Sensors

View full text Add to dashboard Cite

Processes for evaluating software architecture (SA) help to investigate problems and potential risks in SA. It is derived from many studies that proposed a plethora of systematic SA evaluation methods, while industrial practitioners currently refrain from applying them since they are heavyweight. Nowadays, heterogeneous software architectures are organized based on the new infrastructure. Hardware and associated software allow different systems, such as embedded, sensor-based, modern AI, and cloud-based systems, to cooperate efficiently. It brings more complexities to SA evaluation. Alternatively, lightweight architectural evaluation methods have been proposed to satisfy the practitioner’s concerns, but practitioners still do not adopt these methods. This study employs a systematic literature review with a text analysis of SA’s definitions to propose a comparison framework for SA. It identifies lightweight features and factors to improve the architectural evaluation methods among industrial practitioners. The features are determined based on the practitioner’s concerns by analyzing the architecture’s definitions from stakeholders and reviewing architectural evaluation methods. The lightweight factors are acquired by studying the five most commonly used lightweight methods and the Architecture-based Tradeoff Analysis Method (ATAM), the most well-known heavyweight method. Subsequently, the research addresses these features and factors.

show abstract

Architecture to improve the accuracy of automatic image annotation systems

Cited by 10 publications

References 50 publications

Deep Convolutional Neural Network with KNN Regression for Automatic Image Annotation

Deep Convolutional Neural Network with KNN Regression for Automatic Image Annotation

Enhancing high-vocabulary image annotation with a novel attention-based pooling

Lightweight Software Architecture Evaluation for Industry: A Comprehensive Review

Contact Info

Product

Resources

About