Image Retrieval Evaluation Metric for Songket Motif

Abstract

Songket is a fine art heritage specializing in promoting the unique features of Malay identity. Past studies have shown that hundreds of Songket motifs had been produced, but unfortunately, most were not stored digitally. However, the digital collection of image data and determining its ground truth data should be given attention. This paper focuses on an evaluation metric to retrieve image Songket motifs. The initial label for each class of images in the database is ground truth data. The activity of determining the ground truth data involves two research objectives that have been discussed, namely identifying the ground truth data set of Songket Motifs involving Activity One, to obtain two ground truth data sets, precisely the training data set and the test data set involving six categories, to be specific 'Flora', 'Fauna', 'Nature', 'Cosmos', 'Food' and 'Calligraphy'. This phase test was carried out through a survey using a qualitative method, which is participatory by 15 respondents who have classified 413 specific motif images into 56 Songket motifs categories that refer to the six prominent motifs. Meanwhile, Activity Two is a validation-classification test of ground truth data sets by three experts to equate the selection of general and expert respondents to obtain training data sets for testing purposes involving six categories. After rearranging, only 50 ground truth Specific Motifs have been selected. Accordingly, the relationship coefficient correlation method is also implemented to see the relationship between two data through a statistical evaluation angle. In addition, precision and recall methods are also used to obtain precision and recall values for each ground truth data, and the F-measurement method is used to make a single evaluation. The F-measurement result for each category 'Flora': 26.7 -100 (20 ID-Category), 'Fauna': 35.3 -100 (6 ID-Category), 'Nature': 30.8 -100 (5 ID-Category), 'Cosmos': 53.3 -100 (7 ID-Category), and 'Motif': 47.6 -100 (9 ID-Category). Using ground truth data enables image retrieval research to conduct unbiased system testing and evaluation.