Optimal Trade-Off Between Sample Size and Precision of Supervision for the Fixed Effects Panel Data Model

Gnecco, Giorgio; Nutarelli, Federico

doi:10.1007/978-3-030-37599-7_44

Cited by 6 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The opposite occurs for "increasing returns of scale", whereas the case of "constant returns of scale" is intermediate. These results are qualitatively in line with the ones obtained in Gnecco and Nutarelli (2020) for the balanced case and in Gnecco and Nutarelli (2019a, b) for simpler linear regression problems, to which the ordinary/weighted least squares algorithms were applied. This depends on the fact that, in all these cases, the large-sample approximation of the conditional generalization error (or its large-sample upper bound) has the functional form σ 2 T K i , where K i is either a constant, or depends on optimization variables related to neither σ nor T .…”

Section: Remark 41 In Our Previous Conference Work (Gnecco Andsupporting

confidence: 89%

“…The results that will be presented in this paper concerning the unbalanced fixed effects panel data model are consistent with those of Gnecco and Nutarelli (2020) for the balanced case, and those of Gnecco and Nutarelli (2019a, b) concerning simpler linear regression models. Specifically, we show that, also for the unbalanced fixed effects panel data model, the following holds.…”

Section: Introductionsupporting

confidence: 86%

“…The present work extends significantly the analysis of our previous conference article (Gnecco and Nutarelli 2020) to the unbalanced fixed effects panel data model, which is more general than the balanced case considered therein, and leads to an optimization problem that is more complex to investigate. Indeed, in Gnecco and Nutarelli (2020), all the units are always selected at each instant, therefore the selection probabilities do not appear as optimization variables in the corresponding model. Moreover, theoretical arguments are reported in much more details in the current work.…”

Section: Introductionmentioning

confidence: 71%

“…For a total observation time T , denoting by E the expectation operator, the expected number of observations for each unit n is E {T n } = q n T . The balanced case, which was considered in the analysis of Gnecco and Nutarelli (2020), corresponds to the situation q n = 1 for each n. ...,N , t=1,...,T n be a collection of mutually independent and identically distributed random variables, having mean 0 and the same variance σ 2 . Moreover, let all the ε n,t be independent also from all the x n,t .…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Optimal trade-off between sample size, precision of supervision, and selection probabilities for the unbalanced fixed effects panel data model

2020

Self Cite

View full text Add to dashboard Cite

This paper is focused on the unbalanced fixed effects panel data model. This is a linear regression model able to represent unobserved heterogeneity in the data, by allowing each two distinct observational units to have possibly different numbers of associated observations. We specifically address the case in which the model includes the additional possibility of controlling the conditional variance of the output given the input and the selection probabilities of the different units per unit time. This is achieved by varying the cost associated with the supervision of each training example. Assuming an upper bound on the expected total supervision cost and fixing the expected number of observed units for each instant, we analyze and optimize the trade-off between sample size, precision of supervision (the reciprocal of the conditional variance of the output) and selection probabilities. This is obtained by formulating and solving a suitable optimization problem. The formulation of such a problem is based on a large-sample upper bound on the generalization error associated with the estimates of the parameters of the unbalanced fixed effects panel data model, conditioned on the training input dataset. We prove that, under appropriate assumptions, in some cases “many but bad” examples provide a smaller large-sample upper bound on the conditional generalization error than “few but good” ones, whereas in other cases the opposite occurs. We conclude discussing possible applications of the presented results, and extensions of the proposed optimization framework to other panel data models.

show abstract

Section: Remark 41 In Our Previous Conference Work (Gnecco Andsupporting

confidence: 89%

Section: Introductionsupporting

confidence: 86%

Section: Introductionmentioning

confidence: 71%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimal trade-off between sample size, precision of supervision, and selection probabilities for the unbalanced fixed effects panel data model

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…A similar idea was recently used in different contexts inGnecco and Nutarelli (2019);Gnecco et al (2020Gnecco et al ( ,, 2021, where the optimal trade-off between the number of training examples and their precision of supervision was investigated for several machine-learning problems, under a given budget constraint on the total cost of their acquisition.…”

mentioning

confidence: 99%

Simple Models in Complex Worlds: Occam’s Razor and Statistical Learning Theory

2022

Self Cite

View full text Add to dashboard Cite

The idea that “simplicity is a sign of truth”, and the related “Occam’s razor” principle, stating that, all other things being equal, simpler models should be preferred to more complex ones, have been long discussed in philosophy and science. We explore these ideas in the context of supervised machine learning, namely the branch of artificial intelligence that studies algorithms which balance simplicity and accuracy in order to effectively learn about the features of the underlying domain. Focusing on statistical learning theory, we show that situations exist for which a preference for simpler models (as modeled through the addition of a regularization term in the learning problem) provably slows down, instead of favoring, the supervised learning process. Our results shed new light on the relations between simplicity and truth approximation, which are briefly discussed in the context of both machine learning and the philosophy of science.

show abstract