DD-KARB: data-driven compliance to quality by rule based benchmarking

Abstract

The problem of compliance checking and assessment is to ensure that the design or implementation of a system meets some desired properties and complies with some rules or regularities. This problem is a key issue in several human and engineering application domains such as organizational management and e-governance, software and IT industries, and software and systems quality engineering. To deal with this problem, some different approaches and methods have been proposed. In addition to the approaches such as formal methods, mathematical proofs, and logical evaluations, benchmarking can be used for compliance assessment. Naturally, a set of benchmarks can shape an applied solution to compliance assessment. In this paper we propose KARB solution system, i.e. keeping away compliance Anomalies through Rule-based Benchmarking. In fact, in our proposed method the rule-based benchmarking means evaluating the conformity of an under-compliance system to a set of rules. In this solution approach, the under-compliance system is specified symbolically (using formal and logical descriptions). Also, the desired rules are specified formally as the semantic logic in the evaluation process. After reviewing the proposed method, a case study was conducted to demonstrate and analyze the KARB solution. The IR-QUMA study (Iranian Survey on Quality in Messenger Apps) was then conducted to evaluate the quality of some messenger applications. According to the evaluation results, the hybrid DD-KARB method (with a combination of semantics-awareness and data-drivenness) is more effective than solo methods and can compute a good estimation for the messenger application user quality scores. Therefore, DD-KARB can be considered as a method for quality benchmarking in this technical context.