Visual behavior modelling for robotic theory of mind

Chen, Boyuan; Vondrick, Carl; Lipson, Hod

doi:10.1038/s41598-020-77918-x

Cited by 15 publications

(15 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…How to assess this is an open research question. One option is to follow what is done with humans where, for example, AToM is indirectly inferred by inducing actions that reveal the ASI observer's understanding of the state of mind of an agent (Chen et al, 2021b ). Another option is to develop agent assessments based upon ToM tests used to evaluate social intelligence in humans.…”

Section: Socially Intelligent Agentsmentioning

confidence: 99%

See 1 more Smart Citation

Supporting Artificial Social Intelligence With Theory of Mind

Williams

Fiore

Jentsch

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

In this paper, we discuss the development of artificial theory of mind as foundational to an agent's ability to collaborate with human team members. Agents imbued with artificial social intelligence will require various capabilities to gather the social data needed to inform an artificial theory of mind of their human counterparts. We draw from social signals theorizing and discuss a framework to guide consideration of core features of artificial social intelligence. We discuss how human social intelligence, and the development of theory of mind, can contribute to the development of artificial social intelligence by forming a foundation on which to help agents model, interpret and predict the behaviors and mental states of humans to support human-agent interaction. Artificial social intelligence will need the processing capabilities to perceive, interpret, and generate combinations of social cues to operate within a human-agent team. Artificial Theory of Mind affords a structure by which a socially intelligent agent could be imbued with the ability to model their human counterparts and engage in effective human-agent interaction. Further, modeling Artificial Theory of Mind can be used by an ASI to support transparent communication with humans, improving trust in agents, so that they may better predict future system behavior based on their understanding of and support trust in artificial socially intelligent agents.

show abstract

Section: Socially Intelligent Agentsmentioning

confidence: 99%

“…This is generally seen as the ability to recognize and attribute mental states, beliefs, desires, and intentions to others. As a core process in social cognition, it is fundamental to our ability to interact (Chen et al, 2021b ). Said another way, social intelligence can be considered the manifestation of Theory of Mind in adults.…”

Section: Introductionmentioning

confidence: 99%

Supporting Artificial Social Intelligence With Theory of Mind

Williams

Fiore

Jentsch

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…Ideally, these sensory systems could be mapped onto the sensing agent and then applied to probabilistic models of observed agents, thus mimicking the reality of shared experience. Recently, Chen et al reported such a system using a visual behavior robotic model that was able to visualize the future plans of another machine with 98.5% success across four different activities (Chen et al, 2021). If similar strides are made in other robotic sensory paradigms, it is reasonable to imagine an integrative AI system possessing multiple sensory modalities as well as a capacity for mapping these sensory experiences onto observed agents.…”

Section: A Neuroscientific Approach To the Alignment Problemmentioning

confidence: 99%

How to Prevent Robotic Sociopaths: A Neuroscience Approach to Artificial Ethics

Moore¹,

Reggente²,

Vaccaro³

et al. 2022

Preprint

View full text Add to dashboard Cite

Artificial intelligence (AI) is expanding into every niche of human life, organizing our activity, expanding our agency and interacting with us to an exponentially increasing extent. At the same time, AI’s efficiency, complexity and refinement are growing at an accelerating speed. An expanding, ubiquitous intelligence that does not have a means to care about us poses a species-level risk. Justifiably, there is a growing concern with the immediate problem of how to engineer an AI that is aligned with human interests. Computational approaches to the alignment problem currently focus on engineering AI systems to (i) parameterize human values such as harm and flourishing, and (ii) avoid overly drastic solutions, even if these are seemingly optimal. In parallel, ongoing work in applied AI (caregiving, consumer care) is concerned with developing artificial empathy, teaching AI’s to decode human feelings and behavior, and evince appropriate emotional responses.We propose that in the absence of affective empathy (which allows us to share in the states of others), existing approaches to artificial empathy may fail to reliably produce the pro-social, caring component of empathy, potentially resulting in increasingly cognitively complex sociopaths. We adopt the colloquial usage of the term “sociopath” to signify an intelligence possessing cognitive empathy (i.e., the ability to decode, infer, and model the mental and affective states of others), but crucially lacking pro-social, empathic concern arising from shared affect and embodiment. It is widely acknowledged that aversion to causing harm is foundational to the formation of empathy and moral behavior. However, harm aversion is itself predicated on the experience of harm, within the context of the preservation of physical integrity. Following from this, we argue that a “top-down” rule-based approach to achieving caring AI may be inherently unable to anticipate and adapt to the inevitable novel moral/logistical dilemmas faced by an expanding AI. Crucially, it may be more effective to coax caring to emerge from the bottom up, baked into an embodied, vulnerable artificial intelligence with an incentive to preserve its physical integrity. This may be achieved via iterative optimization within a series of tailored environments with incentives and contingencies inspired by the development of empathic concern in humans. Here we attempt an outline of what these training steps might look like. We speculate that work of this kind may allow for AI that surpasses empathic fatigue and the idiosyncrasies, biases, and computational limits that restrict human empathy. While for us, “a single death is a tragedy, a million deaths are a statistic”, the scaleable complexity of AI may allow it to deal proportionately with complex, large-scale ethical dilemmas. Hopefully, by addressing this problem seriously in the early stages of AI’s integration with society, we may one day be accompanied by AI that plans and behaves with a deeply ingrained weight placed on the welfare of others, coupled with the cognitive complexity necessary to understand and solve extraordinary problems.

show abstract

“…Deep learning methods, and in particular convolutional neural networks (CNNs), have been successfully used in a wide range of design applications, from generating car silhouettes to predicting molecule structures [6][7][8][9]. Furthermore, CNNs have the capacity to model complex phenomena, such as classifying objects in images, recognizing actions in videos and predicting the dynamic behaviour of a lava lamp or a candle flame, as well as to predict the behaviour of another agent in a game of hide and seek [10][11][12]. However, in most cases, CNN models are trained on raw image data, without access to user-selected relevant variables or analytical models.…”

Section: Deep Neural Network Models For Visual Intuitionmentioning

confidence: 99%

Visual design intuition: predicting dynamic properties of beams from raw cross-section images

Wyder

Lipson

2021

J. R. Soc. Interface.

Self Cite

View full text Add to dashboard Cite

In this work we aim to mimic the human ability to acquire the intuition to estimate the performance of a design from visual inspection and experience alone. We study the ability of convolutional neural networks to predict static and dynamic properties of cantilever beams directly from their raw cross-section images. Using pixels as the only input, the resulting models learn to predict beam properties such as volume maximum deflection and eigenfrequencies with 4.54% and 1.43% mean average percentage error, respectively, compared with the finite-element analysis (FEA) approach. Training these models does not require prior knowledge of theory or relevant geometric properties, but rather relies solely on simulated or empirical data, thereby making predictions based on ‘experience’ as opposed to theoretical knowledge. Since this approach is over 1000 times faster than FEA, it can be adopted to create surrogate models that could speed up the preliminary optimization studies where numerous consecutive evaluations of similar geometries are required. We suggest that this modelling approach would aid in addressing challenging optimization problems involving complex structures and physical phenomena for which theoretical models are unavailable.

show abstract

Visual behavior modelling for robotic theory of mind

Cited by 15 publications

References 49 publications

Supporting Artificial Social Intelligence With Theory of Mind

Supporting Artificial Social Intelligence With Theory of Mind

How to Prevent Robotic Sociopaths: A Neuroscience Approach to Artificial Ethics

Visual design intuition: predicting dynamic properties of beams from raw cross-section images

Contact Info

Product

Resources

About