Accuracy of Mixed-Source Statistics as Affected by Classification Errors

Delden, Arnout van; Scholtus, Sander; Burger, Joep

doi:10.1515/jos-2016-0032

Cited by 17 publications

(27 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It mostly contains elementary probability manipulations. We make one simplifying assumption compared to [19]: the probabilities p ghi do not depend on i. We write p gh instead and only a single contingency matrix P remains.…”

Section: Methodsmentioning

confidence: 99%

“…In fact, it can be shown that equation (1.2) holds for a multi-class variable s and any numerical variable y, with P being the K × K contingency matrix [19]. We now make two crucial observations.…”

Section: Introductionmentioning

confidence: 94%

“…We partly adopt the notation from [19] to formulate our problem statement. Consider a set I of objects i = 1, 2, .…”

Section: Problem Statementmentioning

confidence: 99%

See 2 more Smart Citations

A Bayesian Approach for Accurate Classification-Based Aggregates

Meertens

Diks²,

Herik³

et al. 2019

Proceedings of the 2019 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

In this paper, we study the accuracy of values aggregated over classes predicted by a classification algorithm. The problem is that the resulting aggregates (e.g., sums of a variable) are known to be biased. The bias can be large even for highly accurate classification algorithms, in particular when dealing with class-imbalanced data. To correct this bias, the algorithm's classification error rates have to be estimated. In this estimation, two issues arise when applying existing bias correction methods. First, inaccuracies in estimating classification error rates have to be taken into account. Second, impermissible estimates, such as a negative estimate for a positive value, have to be dismissed. We show that both issues are relevant in applications where the true labels are known only for a small set of data points. We propose a novel bias correction method using Bayesian inference. The novelty of our method is that it imposes constraints on the model parameters. We show that our method solves the problem of biased classificationbased aggregates as well as the two issues above, in the general setting of multi-class classification. In the empirical evaluation, using a binary classifier on a real-world dataset of company tax returns, we show that our method outperforms existing methods in terms of mean squared error.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 94%

See 1 more Smart Citation

A Bayesian Approach for Accurate Classification-Based Aggregates

Meertens

Diks²,

Herik³

et al. 2019

Proceedings of the 2019 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

show abstract

“…Both reasons imply that the probability of a classification error (more specifically, a false negative classification error) increases as turnover decreases. However, we make the assumption because accurately estimating P for different turnover classes, as suggested by Van Delden et al (2016), requires a far larger training data set than the training data set that we have available.…”

Section: Classification Error Modelmentioning

confidence: 99%

“…If we assume the classification error model, it follows that E.â i / = P T a i and therefore To estimate the bias as given in expression (7), we could use the plug-in estimator B 0 = .P T − I 2 /ŷ: . 8/ Following Van Delden et al (2016), we assume that E.P T / = P T and thatP T andŷ are uncorrelated. It follows that E.B 0 / = P T B.ŷ/; hence the plug-in estimator is a biased estimator of the bias.…”

Section: Estimating Bias and Variancementioning

confidence: 99%

A Data-Driven Supply-Side Approach for Estimating Cross-Border Internet Purchases Within the European Union

Meertens

Diks

Herik

et al. 2019

Journal of the Royal Statistical Society Series A: Statistics in Society

View full text Add to dashboard Cite

Summary The digital economy is a highly relevant item on the European Union's policy agenda. We focus on cross‐border Internet purchases, as part of the digital economy, the total value of which cannot be accurately estimated by using existing consumer survey approaches. In fact, they lead to a serious underestimation. To obtain an accurate estimate, we propose a three‐step data‐driven approach based on supply‐side data. For the first step, we develop a data‐driven generic method for firm level probabilistic record linkage of tax data and business registers. In the second step, we use machine learning to identify webshops based on website data. Then, in the third step, we implement recently developed bias correction techniques that have hitherto been overlooked by the machine learning community. Subsequently, we claim that our three‐step approach can be applied to any European Union member state, leading to more accurate estimates of cross‐border Internet purchases than those obtained by currently existing approaches. To justify the claim, we apply our approach to the Netherlands for the year 2016 and find an estimate that is six times as high as current estimates, having a standard deviation of 8%. Hence, we may conclude that our new approach deserves more investigation and applications.

show abstract

Complementarities of Survey and Population Registers

Zhang

2022

Wiley StatsRef: Statistics Reference Online

View full text Add to dashboard Cite

Survey data collected by sampling or census and register data recorded administratively can be used on an equal footing for creating frames, estimating population parameters, and evaluating statistics. Instead of conducting a population census, census‐like statistics can be produced with less cost by combining survey sampling and population registers. The cost can be further reduced and timeliness improved by fully register‐based approaches, where audit sampling inference can provide valid uncertainty measures.

show abstract

Accuracy of Mixed-Source Statistics as Affected by Classification Errors

Cited by 17 publications

References 19 publications

A Bayesian Approach for Accurate Classification-Based Aggregates

A Bayesian Approach for Accurate Classification-Based Aggregates

A Data-Driven Supply-Side Approach for Estimating Cross-Border Internet Purchases Within the European Union

Complementarities of Survey and Population Registers

Contact Info

Product

Resources

About