Metrics Related to Confusion Matrix as Tools for Conformity Assessment Decisions

Abstract

Conformity assessment refers to activities undertaken to check whether some product, service or process meets certain criteria and specifications given by internationally accepted standards. The decision on whether a property of interest is aligned with the set standards is made based on measurement. However, uncertainty associated with the measurement results may lead to incorrect decisions. Measurement results may be falsely rejected as non-conforming, although they meet specifications. This is referred to as the producer’s risk. If the measurement result that does not meet the required specifications is accepted as conforming, this is referred to as the consumer’s risk. This paper covers calculations of global consumer's and producer's risk using the Bayesian approach and deals with the application of metrics related to confusion matrices in conformity assessment. These techniques have been used to assess the conformity of the bearing ring diameter with the given specifications. Based on the behavior of these metrics, the optimal length of the guard band was determined with the aim of minimizing the global consumer’s and producer’s risk.

Risk Assessment for Linear Regression Models in Metrology

“…In addition, it is valid that the points y j = y x j = x j are the points of the line y = x, which corresponds to the perfect measurement. In other words, from equation (18), it follows that the acceptance interval occupies 80% of the tolerance interval. The 10% of the length of the guard band occupies the area between the acceptance line (curve) A L and the tolerance line (curve) T L , below the line y = x, and 10% of the length of the guard band occupies the area between the acceptance line (curve) A U and the tolerance line (curve) T U , above the line y = x (Figure 4).…”

“…The lower and upper limits of the acceptance interval are defined by introducing a multiplicative factor r ∈ [−1, 1]. An equidistant subdivision of the interval [−1, 1] with a subdivision rate of 0.1 results in 21 subdivision nodes [18]. These subdivision nodes have the form r k = −1 + 0.1•(k − 1), k = 1, 2, .…”

“…In the simplification procedure, which is elaborately outlined in detail in [18], the double integrals present in formulas ( 24) and ( 25) can be reduced to single integrals. The global producer risk for models M1 and M2 can be calculated using the expression:…”

“…The measurement uncertainty that occurs during measurement may lead to incorrect decisions regarding the acceptance of a non-compliant product or the rejection of a product that meets the specifications. This happens when the measurement is close to either the lower limit of the tolerance interval T L or the upper limit of the tolerance interval T U , and the measurement uncertainty associated with that measured value goes over the tolerance interval [T L , T U ] [17][18][19][20][21].…”

Eigenvalue Distributions in Random Confusion Matrices: Applications to Machine Learning Evaluation

“…Confusion matrices are widely used to assess the performance of a classification algorithm by providing a detailed breakdown of the predicted and actual class labels [2][3][4]. The eigenvalues of a confusion matrix offer insights into the underlying structure and characteristics of the classification results [5][6][7][8][9]. Eigenvalues are a mathematical concept used to analyze linear transformations, and in the context of confusion matrices, they can reveal information about the matrix's behaviour [10,11].…”