Progressive Unsupervised Learning of Local Descriptors

Wang, Wufan; Zhang, Lei; Huang, Hua

doi:10.1145/3503161.3547792

Cited by 2 publications

(15 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The second approach is to develop a specialized optimizer that takes the nondifferentiability into account (e.g., Friedman et al, 2010; Gong et al, 2013; Hastie et al, 2015; Huang, 2020a; Huang et al, 2017; Yuan et al, 2010, 2012). These optimizers exploit that the points at which the penalty function is nondifferentiable are well known and result in sparse solutions without the threshold parameter ϵ 2 .…”

Section: Regularized Continuous Time Dynamic Networkmentioning

confidence: 99%

“…In Appendix C we adapt GLMNET for the LASSO regularized CTSEM presented here. The general iterative shrinkage and thresholding algorithm (GIST; Gong et al, 2013) is an alternative to GLMNET and offers solutions for many different penalty functions, which could be used in future applications. Therefore, we also present adaptions of GIST in Appendix C.…”

Section: Regularized Continuous Time Dynamic Networkmentioning

confidence: 99%

“…regCtsem currently supports LASSO regularization, sLASSO regularization, and aLASSO regularization of any parameter toward 0 (or any other value). It offers exact optimization with a variant of GLMNET (Friedman et al, 2010; Yuan et al, 2012) and GIST (Gong et al, 2013; see Appendix C for more details) and approximate optimization using Rsolnp (Ghalanos & Theussl, 2015; Y. Ye, 1987) or optimx (Nash & Varadhan, 2011).…”

Section: Regularized Continuous Time Structural Equation Modeling Wit...mentioning

confidence: 99%

“…In regularized CTSEM we found that GLMNET can sometimes be difficult to set up due to the required Hessian approximation. Therefore, we also implemented the GIST optimizer (Gong et al, 2013). Similar to GLMNET, the general idea is to replace the loss function with an approximation.…”

Section: Full Information Maximum Likelihoodmentioning

confidence: 99%

“…In the following we will present the adaptions necessary for the sLASSO and aLASSO regularization in regCtsem . For details on the optimizers, the interested reader is referred to Friedman et al (2010) and Yuan et al (2012) for GLMNET and Gong et al (2013) for GIST.GLMNET updates all parameters using a combination of outer and inner iterations. In outer iteration k , the parameters are updated with boldθ(k+1)=boldθ(k)+αbold-italicd(k) where α is a step size and bold-italicd(k) a vector with the step direction.…”

Section: Full Information Maximum Likelihoodmentioning

confidence: 99%

See 4 more Smart Citations

Regularized continuous time structural equation models: A network perspective.

Orzek¹,

Voelkle²

2023

Psychological Methods

View full text Add to dashboard Cite

Regularized continuous time structural equation models are proposed to address two recent challenges in longitudinal research: Unequally spaced measurement occasions and high model complexity. Unequally spaced measurement occasions are part of most longitudinal studies, sometimes intentionally (e.g., in experience sampling methods) sometimes unintentionally (e.g., due to missing data). Yet, prominent dynamic models, such as the autoregressive cross-lagged model, assume equally spaced measurement occasions. If this assumption is violated parameter estimates can be biased, potentially leading to false conclusions. Continuous time structural equation models (CTSEM) resolve this problem by taking the exact time point of a measurement into account. This allows for any arbitrary measurement scheme. We combine CTSEM with LASSO and adaptive LASSO regularization. Such regularization techniques are especially promising for the increasingly complex models in psychological research, the most prominent example being network models with often dozens or hundreds of parameters. Here, LASSO regularization can reduce the risk of overfitting and simplify the model interpretation. In this article we highlight unique challenges in regularizing continuous time dynamic models, such as standardization or the optimization of the objective function, and offer different solutions. Our approach is implemented in the R (R Core Team, 2022) package regCtsem. We demonstrate the use of regCtsem in a simulation study, showing that the proposed regularization improves the parameter estimates, especially in small samples. The approach correctly eliminates true-zero parameters while retaining true-nonzero parameters. We present two empirical examples and end with a discussion on current limitations and future research directions.

show abstract

Section: Regularized Continuous Time Dynamic Networkmentioning

confidence: 99%

Section: Regularized Continuous Time Dynamic Networkmentioning

confidence: 99%

Section: Regularized Continuous Time Structural Equation Modeling Wit...mentioning

confidence: 99%

Section: Full Information Maximum Likelihoodmentioning

confidence: 99%

Section: Full Information Maximum Likelihoodmentioning

confidence: 99%

See 3 more Smart Citations

Regularized continuous time structural equation models: A network perspective.

Orzek¹,

Voelkle²

2023

Psychological Methods

View full text Add to dashboard Cite

show abstract

SDGMNet: Statistic-Based Dynamic Gradient Modulation for Local Descriptor Learning

Deng,

2024

AAAI

View full text Add to dashboard Cite

Rescaling the backpropagated gradient of contrastive loss has made significant progress in descriptor learning. However, current gradient modulation strategies have no regard for the varying distribution of global gradients, so they would suffer from changes in training phases or datasets. In this paper, we propose a dynamic gradient modulation, named SDGMNet, for contrastive local descriptor learning. The core of our method is formulating modulation functions with dynamically estimated statistical characteristics. Firstly, we introduce angle for distance measure after deep analysis on backpropagation of pair-wise loss. On this basis, auto-focus modulation is employed to moderate the impact of statistically uncommon individual pairs in stochastic gradient descent optimization; probabilistic margin cuts off the gradients of proportional triplets that have achieved enough optimization; power adjustment balances the total weights of negative pairs and positive pairs. Extensive experiments demonstrate that our novel descriptor surpasses previous state-of-the-art methods in several tasks including patch verification, retrieval, pose estimation, and 3D reconstruction.

show abstract

Progressive Unsupervised Learning of Local Descriptors

Cited by 2 publications

References 40 publications

Regularized continuous time structural equation models: A network perspective.

Regularized continuous time structural equation models: A network perspective.

SDGMNet: Statistic-Based Dynamic Gradient Modulation for Local Descriptor Learning

Contact Info

Product

Resources

About