Flexible feature discovery and structural information control

Kamimura, Ryotaro; Kamimura, Taeko; Uchida, Osamu

doi:10.1080/09540090110108679

Cited by 67 publications

(31 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have so far shown that competitive learning as well as self-organizing maps aim to maximize mutual information between input patterns and output neurons [59], [60], [61]. However, little attention has been paid to information content in input neurons.…”

Section: ) Validity Of Methods and Experimental Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Double Competition for Information-Theoretic SOM

Kamimura¹

2014

ijarai

Self Cite

View full text Add to dashboard Cite

Abstract-In this paper, we propose a new type of informationtheoretic method for the self-organizing maps (SOM), taking into account competition between competitive (output) neurons as well as input neurons. The method is called "double competition", as it considers competition between outputs as well as input neurons. By increasing information in input neurons, we expect to obtain more detailed information on input patterns through the information-theoretic method. We applied the informationtheoretic methods to two well-known data sets from the machine learning database, namely, the glass and dermatology data sets. We found that the information-theoretic method with double competition explicitly separated the different classes. On the other hand, without considering input neurons, class boundaries could not be explicitly identified. In addition, without considering input neurons, quantization and topographic errors were inversely related. This means that when the quantization errors decreased, topographic errors inversely increased. However, with double competition, this inverse relation between quantization and topographic errors was neutralized. Experimental results show that by incorporating information in input neurons, class structure could be clearly identified without degrading the map quality to severely.

show abstract

Section: ) Validity Of Methods and Experimental Resultsmentioning

confidence: 99%

“…In particular, we have introduced informationtheoretic competitive learning [59], [60], [61]. Contrary to the computational methods so far developed, we have supposed that competitive learning is a realization of mutual information maximization between output neurons and input neurons.…”

Section: A Double Competitionmentioning

confidence: 99%

Double Competition for Information-Theoretic SOM

Kamimura¹

2014

ijarai

Self Cite

View full text Add to dashboard Cite

show abstract

“…We have so far demonstrated that competitive processes in competitive learning can be described by using the mutual information between competitive units and input patterns (Kamimura & Kamimura, 2000;Kamimura et al, 2001;Kamimura, 2003a;c;. In other words, the degree of organization of competitive units can be described by using mutual information between competitive units and input patterns.…”

Section: Information-theoretic Competitive Learningmentioning

confidence: 99%

“…First, information contained in competitive units must be as large as possible, as shown in Figure 1(b1). We have already shown that this information on competitive units, more exactly, mutual information between competitive units and input patterns, represents competitive processes (Kamimura & Kamimura, 2000;Kamimura et al, 2001;Kamimura, 2003a;c;. Thus, this information, or more exactly, mutual information, should be as large as possible.…”

Section: Information-theoretic Approachmentioning

confidence: 99%

Information-Theoretic Approach to Interpret Internal Representations of Self-Organizing Maps

Kamimura¹

2011

Self Organizing Maps - Applications and Novel Algorithm Design

Self Cite

View full text Add to dashboard Cite

“…Information maximization methods have long been used to interpret final representations by simplifying network configurations [18,19,20]. In this case, the information content can be stored in a small number of neurons and connection weights.…”

Section: Potential Learning For Interpretationmentioning

confidence: 99%

Self-Assimilation for Solving Excessive Information Acquisition in Potential Learning

Kamimura

Kitago

2017

Journal of Artificial Intelligence and Soft Computing Research

Self Cite

View full text Add to dashboard Cite

The present paper aims to propose a new computational method for potential learning to improve generalization and interpretation. Potential learning has been proposed to simplify the computational procedures of information maximization and to specify which neurons should be fired. However, it is often the case that potential learning sometimes absorbs too much information content on input patterns in the early stage of learning, which tends to degrade generalization performance. This can be solved by making potential learning as slow as possible. Accordingly, we here propose a procedure called "self-assimilation" in which connection weights are accentuated by their characteristics observed in the specific learning step. This makes it possible to predict future connection weights in the early stage of learning. Thus, it is possible to improve generalization by slow learning and at the same time to improve the interpretation of connection weights via the enhanced characteristics of the connection weights. The method was applied to an artificial data set, as well as a real data set of counter services at a local government office in the Tokyo metropolitan area. The results show that improved generalization was observed by making learning as slow as possible. In addition, the number of strong connection weights became smaller for better interpretation by self-assimilation.

show abstract

Flexible feature discovery and structural information control

Cited by 67 publications

References 27 publications

Double Competition for Information-Theoretic SOM

Double Competition for Information-Theoretic SOM

Information-Theoretic Approach to Interpret Internal Representations of Self-Organizing Maps

Self-Assimilation for Solving Excessive Information Acquisition in Potential Learning

Contact Info

Product

Resources

About