The nature of metacognitive inefficiency in perceptual decision making.

Abstract

Humans have the metacognitive ability to judge the accuracy of their own decisions via confidence ratings. A substantial body of research has demonstrated that human metacognition is fallible but it remains unclear how metacognitive inefficiency should be incorporated into a mechanistic model of confidence generation. Here we show that, contrary to what is typically assumed, metacognitive inefficiency depends on the level of confidence. We found that, across 5 different data sets and 4 different measures of metacognition, metacognitive ability decreased with higher confidence ratings. To understand the nature of this effect, we collected a large dataset of 20 subjects completing 2,800 trials each and providing confidence ratings on a continuous scale. The results demonstrated a robustly nonlinear zROC curve with downward curvature, despite a decades-old assumption of linearity. This pattern of results was reproduced by a new mechanistic model of confidence generation, which assumes the existence of lognormally distributed metacognitive noise. The model outperformed competing models either lacking metacognitive noise altogether or featuring Gaussian metacognitive noise. Further, the model could generate a measure of metacognitive ability which was independent of confidence levels. These findings establish an empirically validated model of confidence generation, have significant implications about measures of metacognitive ability, and begin to reveal the underlying nature of metacognitive inefficiency.

Metacognitive improvement: Disentangling adaptive training from experimental confounds.

“…Altogether, these analyses thus suggest that the presence of incentives might be the main reason for the increase in confidence ratings, which in turn would have led to an increase of metacognitive efficiency, as recently proposed (Shekhar & Rahnev, 2021a). Nonetheless, because our analyses relied on comparing confidence biases between studies in relatively small samples, these conclusions on the specific mechanism at stake should be taken with caution.…”

Measuring metacognitive performance: type 1 performance dependence and test-retest reliability

“…While there are some initial proposals for such models ( Bang et al. 2019 ; Shekhar and Rahnev 2021 ), there is currently no established model.…”

A robust confidence–accuracy dissociation via criterion attraction

“…The fitting was performed on the actual stimulus orientations encountered by each individual participant. To find the best fit, I computed the log-likelihood value associated with the full distribution of probabilities of each response type, as done previously ( Yeon and Rahnev 2020 ; Shekhar and Rahnev 2021b ): \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$$\begin{equation*}Log\ likelihood = \mathop \sum \limits_{i,j,k} {\rm{log}}({p_{ijk}})*{n_{ijk}}\end{equation*}$$\end{document} where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}${p_{ijk}}$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}${n_{ijk}}$\end{document} are the response probability and the number of trials, respectively, associated with the Distribution \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$i \in \left\{ {1,2} \right\}$\end{document} , confidence rating \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$j \in \left\{ {1,2,3,4} \right\}$\end{document} , and condition \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$k$\end{document} , where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$k = 1$\end{document} corresponds to the low variability condition and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$k = 2$\end{document} corresponds to the high variability condition. The best fit was determined as the set of parameters that maximized the log-likelihood value.…”