Analysis Dialogs and Machine Consciousness

Abstract

Analysis dialogs aim at analyzing the operation of a chatbot or more generally of a question answering system to discover its limitations and maybe discover their nonhuman nature as in the case of the Turing test. The answers elicited from the system may be accompanied by explanations that are crucial for judging whether a system is self-aware. Self-awareness of question answering systems, or the so-called “artificial consciousness” require the recording of the actions that a system performs to generate its answer. These actions may be represented either as a path of state changes or as a sequence of reasoning steps. When this path or sequence is too long, an analysis dialog may aim at exploring the capability of a system to summarize the raw explanations and generate shorter explanations friendlier to the interrogating user. The real analysis dialogs of two Turing test champions, namely Chip Vivant and Mitsuku with the user are presented and commented on. The comments aim at clarifying the difficulty of these systems to answer reasonably some questions a fact that indicates their nonhuman nature. The methodology tested was applied to ChatGPT, and the results are presented with analogous comments. An appropriate subset of questions augmented by new ones was used.