Multiway Monte Carlo Method for Linear Systems

Wu, Tao; Gleich, David F.

doi:10.1137/18m121527x

Cited by 6 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where N O is the number of out-of-distribution samples drawn from P O (x, y). Based on Monte Carlo [32], we apply the stochastic gradient descent optimization algorithm [1] to estimate the gradients of Eq. ( 13) and Eq.…”

Section: Generic Empirical Riskmentioning

confidence: 99%

Out-of-distribution Detection by Cross-class Vicinity Distribution of In-distribution Data

Zhao¹,

Cao²,

Lin³

2022

Preprint

View full text Add to dashboard Cite

Deep neural networks only learn to map indistribution inputs to their corresponding ground truth labels in the training phase without differentiating out-of-distribution samples from in-distribution ones. This results from the assumption that all samples are independent and identically distributed without distributional distinction. Therefore, a pretrained network learned from the in-distribution samples treats out-ofdistribution samples as in-distribution and makes high-confidence predictions on them in the test phase. To address this issue, we draw out-of-distribution samples from the vicinity distribution of training in-distribution samples for learning to reject the prediction on out-of-distribution inputs. A Cross-class Vicinity Distribution is introduced by assuming that an out-of-distribution sample generated by mixing multiple in-distribution samples does not share the same classes of its constituents. We thus improve the discriminability of a pretrained network by finetuning it with out-of-distribution samples drawn from the cross-class vicinity distribution, where each out-of-distribution input corresponds to a complementary label. Experiments on various in-/out-ofdistribution datasets show that the proposed method significantly outperforms existing methods in improving the capacity of discriminating between in-and out-of-distribution samples.

show abstract

Section: Generic Empirical Riskmentioning

confidence: 99%

Out-of-distribution Detection by Cross-class Vicinity Distribution of In-distribution Data

Zhao¹,

Cao²,

Lin³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The parameter φ is fixed for learning the parameter θ in f θ because g φ is a pretrained network. Based on Monte Carlo [36], we apply the stochastic gradient descent optimization algorithm [37] to estimate the gradient of Eq. ( 13), where the batch size is B.…”

Section: Learning the Auxiliary Networkmentioning

confidence: 99%

Label and Distribution-discriminative Dual Representation Learning for Out-of-Distribution Detection

Zhao¹,

Cao²

2022

Preprint

View full text Add to dashboard Cite

To classify in-distribution samples, deep neural networks learn label-discriminative representations, which, however, are not necessarily distribution-discriminative according to the information bottleneck. Therefore, trained networks could assign unexpected high-confidence predictions to out-ofdistribution samples drawn from distributions differing from that of in-distribution samples. Specifically, networks extract the strongly label-related information from in-distribution samples to learn the label-discriminative representations but discard the weakly label-related information. Accordingly, networks treat out-of-distribution samples with minimum labelsensitive information as in-distribution samples. According to the different informativeness properties of in-and out-ofdistribution samples, a Dual Representation Learning (DRL) method learns distribution-discriminative representations that are weakly related to the labeling of in-distribution samples and combines label-and distribution-discriminative representations to detect out-of-distribution samples. For a label-discriminative representation, DRL constructs the complementary distributiondiscriminative representation by an implicit constraint, i.e., integrating diverse intermediate representations where an intermediate representation less similar to the label-discriminative representation owns a higher weight. Experiments show that DRL outperforms the state-of-the-art methods for out-of-distribution detection.

show abstract

“…According to the idea of Monte Carlo [47], we apply the stochastic gradient descent (SGD) [43] optimization algorithm to estimate the gradient of the objective function Eq. (15).…”

Section: Confidence Penalty On Out-of-distribution Samplesmentioning

confidence: 99%

Revealing the Distributional Vulnerability of Discriminators by Implicit Generators

Zhao¹,

Cao²,

Lin³

2021

Preprint

View full text Add to dashboard Cite

An explicit discriminator trained on observable in-distribution (ID) samples can make high-confidence prediction on out-of-distribution (OOD) samples due to its distributional vulnerability. This is primarily caused by the limited ID samples observable for training discriminators when OOD samples are unavailable. To address this issue, the state-of-the-art methods train the discriminator with OOD samples generated by general assumptions without considering the data and network characteristics. However, different network architectures and training ID datasets may cause diverse vulnerabilities, and the generated OOD samples thus usually misaddress the specific distributional vulnerability of the explicit discriminator. To reveal and patch the distributional vulnerabilities, we propose a novel method of fine-tuning explicit discriminators by implicit generators (FIG). According to the Shannon entropy, an explicit discriminator can construct its corresponding implicit generator to generate specific OOD samples without extra training costs. A Langevin Dynamic sampler then draws high-quality OOD samples from the generator to reveal the vulnerability. Finally, a regularizer, constructed according to the design principle of the implicit generator, patches the distributional vulnerability by encouraging those generated OOD samples with high entropy. Our experiments on four networks, four ID datasets and seven OOD datasets demonstrate that FIG achieves state-of-the-art OOD detection performance and maintains a competitive classification capability.

show abstract

Multiway Monte Carlo Method for Linear Systems

Cited by 6 publications

References 27 publications

Out-of-distribution Detection by Cross-class Vicinity Distribution of In-distribution Data

Out-of-distribution Detection by Cross-class Vicinity Distribution of In-distribution Data

Label and Distribution-discriminative Dual Representation Learning for Out-of-Distribution Detection

Revealing the Distributional Vulnerability of Discriminators by Implicit Generators

Contact Info

Product

Resources

About