Black-Box-Based Mathematical Modelling of Machine Intelligence Measuring

Abstract

Current machine intelligence metrics rely on a different philosophy, hindering their effective comparison. There is no standardization of what is machine intelligence and what should be measured to quantify it. In this study, we investigate the measurement of intelligence from the viewpoint of real-life difficult-problem-solving abilities, and we highlight the importance of being able to make accurate and robust comparisons between multiple cooperative multiagent systems (CMASs) using a novel metric. A recent metric presented in the scientific literature, called MetrIntPair, is capable of comparing the intelligence of only two CMASs at an application. In this paper, we propose a generalization of that metric called MetrIntPairII. MetrIntPairII is based on pairwise problem-solving intelligence comparisons (for the same problem, the problem-solving intelligence of the studied CMASs is evaluated experimentally in pairs). The pairwise intelligence comparison is proposed to decrease the necessary number of experimental intelligence measurements. MetrIntPairII has the same properties as MetrIntPair, with the main advantage that it can be applied to any number of CMASs conserving the accuracy of the comparison, while it exhibits enhanced robustness. An important property of the proposed metric is the universality, as it can be applied as a black-box method to intelligent agent-based systems (IABSs) generally, not depending on the aspect of IABS architecture. To demonstrate the effectiveness of the MetrIntPairII metric, we provide a representative experimental study, comparing the intelligence of several CMASs composed of agents specialized in solving an NP-hard problem.

Topic Identification in Voice Recordings

“…A possible research direction can be to extend the experimental evaluations and apply some advanced analysis of the experimental results based on the methods presented in the scientific literature, like: [10,11] comparison of two samples of experimental evaluation results; [12,13] comparison of any number of samples of experimental evaluation results ( [12] based on measurements made in pairs, [13] without measurements made in pairs); [14] identification of statistical extremes in the experimental evaluations that could alter significantly the experimental evaluation results.…”

Human-Centric Functional Computing as an Approach to Human-Like Computation

“…On reviewing various measures of AI performance in the literature to determine whether they are consistent with this prediction of being correlated with narrow problem-solving ability, all such assessments that were reviewed were found to be too deeply entangled with the specifics of their own methodology to be easily comparable to the more abstract prediction presented here (Iantovics, 2021), (Safari, 2021). This observation reinforces the importance of a universal, human-centric, and functional only model of AI and artificial cognition that is independent of any implementations of any AI or AGI tools or methods, and therefore doesn't require any expert knowledge that might be required by those tools, such as understanding whatever assumptions they make outside of the assumptions that can be proven to be true or false by any individual within their own experience.…”