A new digital watermarking evaluation and benchmarking methodology using an external group of evaluators and multi‐criteria analysis based on ‘large‐scale data’

Abstract

Summary Digital watermarking evaluation and benchmarking are challenging tasks because of multiple evaluation and conflicting criteria. A few approaches have been presented to implement digital watermarking evaluation and benchmarking frameworks. However, these approaches still possess a number of limitations, such as fixing several attributes on the account of other attributes. Well‐known benchmarking approaches are limited to robust watermarking. Therefore, this paper presents a new methodology for digital watermarking evaluation and benchmarking based on large‐scale data by using external evaluators and a group decision making context. Two experiments are performed. In the first experiment, a noise gate‐based digital watermarking approach is developed, and the scheme for the noise gate digital watermarking approach is enhanced. Sixty audio samples from different audio styles are tested with two algorithms. A total of 120 samples were evaluated according to three different metrics, namely, quality, payload, and complexity, to generate a set of digital watermarking samples. In the second experiment, the situation in which digital watermarking evaluators have different preferences is discussed. Weight measurement with a decision making solution is required to solve this issue. The analytic hierarchy process is used to measure evaluator preference. In the decision making solution, the technique for order of preference by similarity to the ideal solution with different contexts (e.g., individual and group) is utilized. Therefore, selecting the proper context with different aggregation operators to benchmark the results of experiment 1 (i.e., digital watermarking approaches) is recommended. The findings of this research are as follows: (1) group and individual decision making provide the same result in this case study. However, in the case of selection where the priority weights are generated from the evaluators, group decision making is the recommended solution to solve the trade‐off reflected in the benchmarking process for digital watermarking approaches. (2) Internal and external aggregations show that the enhanced watermarking approach demonstrates better performance than the original watermarking approach. © 2016 The Authors. Software: Practice and Experience published by John Wiley & Sons Ltd.

Helping doctors hasten COVID-19 treatment: Towards a rescue framework for the transfusion of best convalescent plasma to the most critical patients based on biological requirements via ml and novel MCDM methods

“…Firstly, the prioritisation is dependant on important attributes; thus, patient and CP selection based on multiple criteria is a multi-attribute decision matrix [57][58][59][60][61]. Secondly, different values of importance are often given for each attribute, which further increases the complexity of the task [63][64][65][66][67]. Finally, a prioritisation process requires synchronised consideration of the inverse relationship amongst the mentioned criteria; thus, a trade-off is created [ 4 9 , 6 8 ], [122].…”

Real-Time Remote-Health Monitoring Systems: a Review on Patients Prioritisation for Multiple-Chronic Diseases, Taxonomy Analysis, Concerns and Solution Procedure

“…MCDM is considered one of the common techniques in decision-making, and it handles the problems depending on available criteria [222][223][224][225]. MCDM involves structuring, planning and solving decision problems by utilising numerous criteria [222,226,227]. The purpose of MCDM is to support decision-makers to solve problems with numerous criteria and alternatives [228][229][230].…”

Assessment and Ranking Framework for the English Skills of Pre-Service Teachers Based on Fuzzy Delphi and TOPSIS Methods

“…To the best of our knowledge, none of these methods have been used to develop a framework to assess and rank pre-service teacher English skills. According to literature [88]- [93], the drawbacks, benefits and recommendations for popular MCDM techniques can be summarised as follows. HAW and WSM techniques are easy to use and understand, but the weights of the attributes are arbitrarily assigned.…”