No abstract

No abstract

We outline eight characteristics of the environments, tasks, and agents important for human-level intelligence. Treating these characteristics as influences on desired agent behavior, we then derive twelve requirements for general cognitive architectures. Cognitive-architecture designs that meet the requirements should support human-level behavior across a wide range of tasks, embedded in environment similar to the real world. Although requirements introduced here are hypothesized as necessary ones for human-level intelligence, our assumption is the list is not yet sufficient to guarantee the achievement of human-level intelligence when met. However, attempts to be explicit about influences and specific requirements may be more productive than direct comparison of architectural designs and features for communication and interaction about cognitive architectures.

This paper represents a first step in attempting to engage the research community in discussions about evaluation of human-level intelligent systems. First, we discuss the challenges of evaluating human-level intelligent systems. Second, we explore the different types of claims that are made about HLI systems, which are the basis for confirmatory evaluations. Finally, we briefly discuss a range of experimental designs that support the evaluation of claims.

No abstract