The Unbearable Shallow Understanding of Deep Learning

Plebe, Alessio; Grasso, G.

doi:10.1007/s11023-019-09512-8

Cited by 30 publications

(16 citation statements)

References 152 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is proposed here that training such networks with different transforms of objects would much better enable transform-invariant shape-based representations to be learned, leading to much more powerful performance. Potential limitations of current deep learning methods have been also been noted by others (Plebe and Grasso, 2019;Sejnowski, 2020).…”

Section: Discussionmentioning

confidence: 81%

Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning

Rolls

2021

Front. Comput. Neurosci.

View full text Add to dashboard Cite

First, neurophysiological evidence for the learning of invariant representations in the inferior temporal visual cortex is described. This includes object and face representations with invariance for position, size, lighting, view and morphological transforms in the temporal lobe visual cortex; global object motion in the cortex in the superior temporal sulcus; and spatial view representations in the hippocampus that are invariant with respect to eye position, head direction, and place. Second, computational mechanisms that enable the brain to learn these invariant representations are proposed. For the ventral visual system, one key adaptation is the use of information available in the statistics of the environment in slow unsupervised learning to learn transform-invariant representations of objects. This contrasts with deep supervised learning in artificial neural networks, which uses training with thousands of exemplars forced into different categories by neuronal teachers. Similar slow learning principles apply to the learning of global object motion in the dorsal visual system leading to the cortex in the superior temporal sulcus. The learning rule that has been explored in VisNet is an associative rule with a short-term memory trace. The feed-forward architecture has four stages, with convergence from stage to stage. This type of slow learning is implemented in the brain in hierarchically organized competitive neuronal networks with convergence from stage to stage, with only 4-5 stages in the hierarchy. Slow learning is also shown to help the learning of coordinate transforms using gain modulation in the dorsal visual system extending into the parietal cortex and retrosplenial cortex. Representations are learned that are in allocentric spatial view coordinates of locations in the world and that are independent of eye position, head direction, and the place where the individual is located. This enables hippocampal spatial view cells to use idiothetic, self-motion, signals for navigation when the view details are obscured for short periods.

show abstract

Section: Discussionmentioning

confidence: 81%

Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning

Rolls

2021

Front. Comput. Neurosci.

View full text Add to dashboard Cite

show abstract

“…But once these complex mappings are learned, their success or failure in a new situation on a given trial cannot be evaluated and corrected by the network. Indeed, the complex mappings achieved by such networks (e.g., networks trained by backpropagation of errors or by reinforcement learning) mean that after training they operate according to fixed rules, and are often quite impenetrable, inflexible, and difficult to correct quickly (Rumelhart et al, 1986;Rolls, 2016;Plebe and Grasso, 2019). In contrast, to correct a multistep, single occasion, syntactically based plan, recall of the steps just made in the reasoning or planning, and perhaps of related episodic material, needs to occur, so that the step in the chain of reasoning that is most likely to be in error can be identified.…”

Section: Higher Order Thought (Hot) Theoriesmentioning

confidence: 99%

Neural Computations Underlying Phenomenal Consciousness: A Higher Order Syntactic Thought Theory

Rolls

2020

Front. Psychol.

View full text Add to dashboard Cite

Problems are raised with the global workspace hypothesis of consciousness, for example about exactly how global the workspace needs to be for consciousness to suddenly be present. Problems are also raised with Carruthers's (2019) version that excludes conceptual (categorical or discrete) representations, and in which phenomenal consciousness can be reduced to physical processes, with instead a different levels of explanation approach to the relation between the brain and the mind advocated. A different theory of phenomenal consciousness is described, in which there is a particular computational system involved in which Higher Order Syntactic Thoughts are used to perform credit assignment on first order thoughts of multiple step plans to correct them by manipulating symbols in a syntactic type of working memory. This provides a good evolutionary reason for the evolution of this kind of computational module, with which, it is proposed, phenomenal consciousness is associated. Some advantages of this HOST approach to phenomenal consciousness are then described with reference not only to the global workspace approach, but also to Higher Order Thought (HOT) theories. It is hypothesized that the HOST system which requires the ability to manipulate first order symbols in working memory might utilize parts of the prefrontal cortex implicated in working memory, and especially the left inferior frontal gyrus, which is involved in language and probably syntactical processing. Overall, the approach advocated is to identify the computations that are linked to consciousness, and to analyze the neural bases of those computations.

show abstract

“…In the last decade, a subfield of Machine Learning named Deep Learning, based on the extensive use of Artificial Neural Networks with high number of layers and artificial neurons led to significant improvements on several data analysis fields such as text analysis, audio and image processing and recognition. In particular, the rapid diffusion of Deep Learning techniques applied to Computer Vision techniques allowed the rise of performances on several visual tasks (Plebe & Grasso, 2019), including the food intake estimation problem. One of the first Deep Learning approach applied to this field has been presented by Meyers et al (2015), that proposed a system named Im2Calories in 2015.…”

Section: Reviewmentioning

confidence: 99%

A review on food recognition technology for health applications

Allegra¹,

Battiato²,

Ortis³

et al. 2020

Health Psych Res

View full text Add to dashboard Cite

Food understanding from digital media has become a challenge with important applications in many different domains. On the other hand, food is a crucial part of human life since the health is strictly affected by diet. The impact of food in people life led Computer Vision specialists to develop new methods for automatic food intake monitoring and food logging. In this review paper we provide an overview about automatic food intake monitoring, by focusing on technical aspects and Computer Vision works which solve the main involved tasks (i.e., classification, recognitions, segmentation, etc.). Specifically, we conducted a systematic review on main scientific databases, including interdisciplinary databases (i.e., Scopus) as well as academic databases in the field of computer science that focus on topics related to image understanding (i.e., recognition, analysis, retrieval). The search queries were based on the following key words: “food recognition”, “food classification”, “food portion estimation”, “food logging” and “food image dataset”. A total of 434 papers have been retrieved. We excluded 329 works in the first screening and performed a new check for the remaining 105 papers. Then, we manually added 5 recent relevant studies. Our final selection includes 23 papers that present systems for automatic food intake monitoring, as well as 46 papers which addressed Computer Vision tasks related food images analysis which we consider essential for a comprehensive overview about this research topic. A discussion that highlights the limitations of this research field is reported in conclusions.

show abstract

The Unbearable Shallow Understanding of Deep Learning

Cited by 30 publications

References 152 publications

Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning

Learning Invariant Object and Spatial View Representations in the Brain Using Slow Unsupervised Learning

Neural Computations Underlying Phenomenal Consciousness: A Higher Order Syntactic Thought Theory

A review on food recognition technology for health applications

Contact Info

Product

Resources

About