Safety Engineering for Artificial General Intelligence

Yampolskiy, Roman V.; Fox, Joshua

doi:10.1007/s11245-012-9128-9

Cited by 51 publications

(53 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Perhaps access to external information can be used to mediate speed of RSI process. This also has significant implications on safety mechanisms we can employ while experimenting with early RSI systems [55][56][57][58][59][60][61][62][63]. Finally, it needs to be investigated if the whole RSI process can be paused at any point and for any specific duration of time in order to limit any negative impact from potential intelligence explosion.…”

Section: Other Propertiesmentioning

confidence: 99%

Analysis of Types of Self-Improving Software

Yampolskiy

2015

Artificial General Intelligence

View full text Add to dashboard Cite

show abstract

Section: Other Propertiesmentioning

confidence: 99%

Analysis of Types of Self-Improving Software

Yampolskiy

2015

Artificial General Intelligence

View full text Add to dashboard Cite

show abstract

“…This initially appears somehow pedestrian, but incorporating the objections of Yampolskiy (2013); Yampolskiy and Fox (2013), it becomes more logical: Yampolskiy and Fox consider that AI may become equivalent to, or exceeds human intelligence level. Reaching human level implicates that AI becomes capable to reproduce and improve its own kind.…”

Section: The Psychodynamic Structure Modelmentioning

confidence: 99%

Safety of Autonomous Cognitive-oriented Robots

Ertle¹

2015

Künstl Intell

View full text Add to dashboard Cite

Service robots shall very soon autonomously provide services in all spheres of life. They have to execute demanding and complex tasks in a dynamic environment, collaborate with human users in a natural and intuitive way and adapt themselves to varying conditions. It can be assumed that complex robots have to be able to learn, if they shall be able to provide complex tasks in unstructured environments in an autonomous fashion. The capability to 'act autonomously' is often mentioned in conjunction with robots, however, the perception and understanding of the term autonomy varies among the different research fields. Therefore, a closer look is taken at robot autonomy and intelligence, in particular, with regard to current and future robots. From this perspective, implications for safety are derived concerning safe autonomous behavior.In order to push forward the robot safety in the light of safe behavior in complex environments, a novel classification of robot hazards is provided. Based on this, the so-called object interaction hazards are derived which arise when objects that are, for instance, located in the near environment, interact with objects that are manipulated by a robot. Taking into account the current state-of-the-art, it can be stated that this denotes a novel problem area. This problem area is so far addressed neither in current research work, nor in the relevant standards.The new type of hazards can be assigned to a group of hazards that originate from the interaction with a complex and unstructured environment. In order to sufficiently consider the environment and operation context, the robot has to be aware of it. In the field of cognitive (technical and biological) systems, this key capability can be called 'situation awareness ' (cf. Söffker, 2008). Based on Endsley (1995)'s definition of situation awareness, Wardziński (2008) proposes the 'dynamic risk assessment' approach, which shall enable the robot perceive the risk of current and upcoming situations. In order to realize such a risk-aware planning system for the first time, dynamic risk assessment is integrated within a cognitive architecture in order to utilize cognitive functions, such as anticipation, planning and learning. Here, the so-called action spaces (sets of possible upcoming situations) are dynamically anticipated within the underlying cognitive architecture, and a risk assessment component assesses them with regard to comprised risks. Thus, the generated risk information can be utilized for a risk-aware action planning. The proper operation of this integrating concept is demonstrated via simulations and with a robot experiment.In order to consider (object interaction) hazards by means of dynamic risk assessment, (initial) knowledge about hazards is required. Thus, a novel procedural model is developed for systematically generating a safety knowledge base. In this connection, the concept is to formalize risk models (risk description rules) in a generalized manner so that they remain valid for future, and so far unknown si...

show abstract

“…Whereas most literature on politicized skepticism (and similar concepts such as denial) is backward-looking, consisting of historical analysis of skepticisms that have already occurred [1,2,[4][5][6][7], this paper is largely (but not exclusively) forward-looking, consisting of prospective analysis of skepticisms that could occur at some point in the future. Meanwhile, the superintelligence governance literature has looked mainly at institutional regulations to prevent research groups from building dangerous superintelligence and support for research on safety measures [8][9][10][11]. This paper contributes to a smaller literature on the role of corporations in superintelligence development [12] and on social and psychological aspects of superintelligence governance [13].…”

Section: Introductionmentioning

confidence: 99%

Superintelligence Skepticism as a Political Tool

Baum¹

2018

Information

View full text Add to dashboard Cite

This paper explores the potential for skepticism about artificial superintelligence to be used as a tool for political ends. Superintelligence is AI that is much smarter than humans. Superintelligence does not currently exist, but it has been proposed that it could someday be built, with massive and potentially catastrophic consequences. There is substantial skepticism about superintelligence, including whether it will be built, whether it would be catastrophic, and whether it is worth current attention. To date, superintelligence skepticism appears to be mostly honest intellectual debate, though some of it may be politicized. This paper finds substantial potential for superintelligence skepticism to be (further) politicized, due mainly to the potential for major corporations to have a strong profit motive to downplay concerns about superintelligence and avoid government regulation. Furthermore, politicized superintelligence skepticism is likely to be quite successful, due to several factors including the inherent uncertainty of the topic and the abundance of skeptics. The paper's analysis is based on characteristics of superintelligence and the broader AI sector, as well as the history and ongoing practice of politicized skepticism on other science and technology issues, including tobacco, global warming, and industrial chemicals. The paper contributes to literatures on politicized skepticism and superintelligence governance.

show abstract

Safety Engineering for Artificial General Intelligence

Cited by 51 publications

References 49 publications

Analysis of Types of Self-Improving Software

Analysis of Types of Self-Improving Software

Safety of Autonomous Cognitive-oriented Robots

Superintelligence Skepticism as a Political Tool

Contact Info

Product

Resources

About