Lie group impression for deep learning

Yang, Mengduo; Li, Fanzhang; Zhang, Li; Zhang, Zhao

doi:10.1016/j.ipl.2018.03.006

Cited by 5 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Anchors are multiple bounding boxes with different sizes and aspect ratios that are generated centered on each pixel of the image. Yang et al (2018) propose a dynamic mechanism named MetaAnchor, which can select appropriate anchor for dynamic generation. Zhang et al (2017) propose a scale compensation anchor matching mechanism to improve the recall rate for tiny objects.…”

Section: Anchor-based Optimizationmentioning

confidence: 99%

Learning geometric Jensen-Shannon divergence for tiny object detection in remote sensing images

Ni,

Lin,

Wang

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

Tiny objects in remote sensing images only have a few pixels, and the detection difficulty is much higher than that of regular objects. General object detectors lack effective extraction of tiny object features, and are sensitive to the Intersection-over-Union (IoU) calculation and the threshold setting in the prediction stage. Therefore, it is particularly important to design a tiny-object-specific detector that can avoid the above problems. This article proposes the network JSDNet by learning the geometric Jensen-Shannon (JS) divergence representation between Gaussian distributions. First, the Swin Transformer model is integrated into the feature extraction stage as the backbone to improve the feature extraction capability of JSDNet for tiny objects. Second, the anchor box and ground-truth are modeled as two two-dimensional (2D) Gaussian distributions, so that the tiny object is represented as a statistical distribution model. Then, in view of the sensitivity problem faced by the IoU calculation for tiny objects, the JSDM module is designed as a regression sub-network, and the geometric JS divergence between two Gaussian distributions is derived from the perspective of information geometry to guide the regression prediction of anchor boxes. Experiments on the AI-TOD and DOTA datasets show that JSDNet can achieve superior detection performance for tiny objects compared to state-of-the-art general object detectors.

show abstract

Section: Anchor-based Optimizationmentioning

confidence: 99%

Learning geometric Jensen-Shannon divergence for tiny object detection in remote sensing images

Ni,

Lin,

Wang

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…Yang et al [41] proposed an algorithm for capturing the Lie group manifold structure of visual impression. They developed single-layer Lie group models and stacked them to yield a deep architecture; they also solved the learning problem of network weight by designing a Lie group-based gradient descent algorithm.…”

Section: Lie Group Deep Structure Learningmentioning

confidence: 99%

Survey on lie group machine learning

Mei¹,

Li²

2020

Big Data Min. Anal.

Self Cite

View full text Add to dashboard Cite

Lie group machine learning is recognized as the theoretical basis of brain intelligence, brain learning, higher machine learning, and higher artificial intelligence. Sample sets of Lie group matrices are widely available in practical applications. Lie group learning is a vibrant field of increasing importance and extraordinary potential and thus needs to be developed further. This study aims to provide a comprehensive survey on recent advances in Lie group machine learning. We introduce Lie group machine learning techniques in three major categories: supervised Lie group machine learning, semisupervised Lie group machine learning, and unsupervised Lie group machine learning. In addition, we introduce the special application of Lie group machine learning in image processing. This work covers the following techniques: Lie group machine learning model, Lie group subspace orbit generation learning, symplectic group learning, quantum group learning, Lie group fiber bundle learning, Lie group cover learning, Lie group deep structure learning, Lie group semisupervised learning, Lie group kernel learning, tensor learning, frame bundle connection learning, spectral estimation learning, Finsler geometric learning, homology boundary learning, category representation learning, and neuromorphic synergy learning. Overall, this survey aims to provide an insightful overview of state-of-the-art development in the field of Lie group machine learning. It will enable researchers to comprehensively understand the state of the field, identify the most appropriate tools for particular applications, and identify directions for future research.

show abstract

“…The authors extend this theory to a more generic case by treating a linear layer as an orthogonal-constrained Stiefel manifold and use a Lie group to transform the calculation process into a linear approximation. The approach has been applied to an image classification problem in research by Yang et al [ 32 ] and showed a significant improvement in the optimization step. Inspired by the idea of the method by Nishimori et al, the proposed method in this research constrains a network to Stiefel manifold so that a meta-learner could perform a more stable gradient descent in limited steps and accelerate the adapting process.…”

Section: Related Workmentioning

confidence: 99%

Multi-Stage Meta-Learning for Few-Shot with Lie Group Network Constraint

Dong

Liu

2020

Entropy

Self Cite

View full text Add to dashboard Cite

Deep learning has achieved many successes in different fields but can sometimes encounter an overfitting problem when there are insufficient amounts of labeled samples. In solving the problem of learning with limited training data, meta-learning is proposed to remember some common knowledge by leveraging a large number of similar few-shot tasks and learning how to adapt a base-learner to a new task for which only a few labeled samples are available. Current meta-learning approaches typically uses Shallow Neural Networks (SNNs) to avoid overfitting, thus wasting much information in adapting to a new task. Moreover, the Euclidean space-based gradient descent in existing meta-learning approaches always lead to an inaccurate update of meta-learners, which poses a challenge to meta-learning models in extracting features from samples and updating network parameters. In this paper, we propose a novel meta-learning model called Multi-Stage Meta-Learning (MSML) to post the bottleneck during the adapting process. The proposed method constrains a network to Stiefel manifold so that a meta-learner could perform a more stable gradient descent in limited steps so that the adapting process can be accelerated. An experiment on the mini-ImageNet demonstrates that the proposed method reached a better accuracy under 5-way 1-shot and 5-way 5-shot conditions.

show abstract

Lie group impression for deep learning

Cited by 5 publications

References 9 publications

Learning geometric Jensen-Shannon divergence for tiny object detection in remote sensing images

Learning geometric Jensen-Shannon divergence for tiny object detection in remote sensing images

Survey on lie group machine learning

Multi-Stage Meta-Learning for Few-Shot with Lie Group Network Constraint

Contact Info

Product

Resources

About