Improved Sampling‐Importance Resampling and Reduced Bias Importance Sampling

Skare, Øivind; Bølviken, Erik; Holden, Lars

doi:10.1111/1467-9469.00360

Cited by 55 publications

(34 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, simple re-weighting is difficult or impossible to apply in the context of many purely data-analytic procedures such as multidimensional scaling or hierarchical clustering. Simulated Importance Resampling [34] provides a useful alternative for RWRW samples, but suffers from well-known problems of asymptotic bias (see [35] for a discussion and some palliatives). This is of less concern for applications such as moment estimation, for which reweighting is both simple and effective.…”

Section: Findings and Practical Recommendationsmentioning

confidence: 99%

Walking in Facebook: A Case Study of Unbiased Sampling of OSNs

Gjoka

Kurant

Butts

et al. 2010

2010 Proceedings IEEE INFOCOM

547

507

View full text Add to dashboard Cite

Abstract-With more than 250 million active users [1], Facebook (FB) is currently one of the most important online social networks. Our goal in this paper is to obtain a representative (unbiased) sample of Facebook users by crawling its social graph. In this quest, we consider and implement several candidate techniques. Two approaches that are found to perform well are the Metropolis-Hasting random walk (MHRW) and a re-weighted random walk (RWRW). Both have pros and cons, which we demonstrate through a comparison to each other as well as to the "ground-truth" (UNI -obtained through true uniform sampling of FB userIDs). In contrast, the traditional Breadth-First-Search (BFS) and Random Walk (RW) perform quite poorly, producing substantially biased results. In addition to offline performance assessment, we introduce online formal convergence diagnostics to assess sample quality during the data collection process. We show how these can be used to effectively determine when a random walk sample is of adequate size and quality for subsequent use (i.e., when it is safe to cease sampling). Using these methods, we collect the first, to the best of our knowledge, unbiased sample of Facebook. Finally, we use one of our representative datasets, collected through MHRW, to characterize several key properties of Facebook.

show abstract

Section: Findings and Practical Recommendationsmentioning

confidence: 99%

Walking in Facebook: A Case Study of Unbiased Sampling of OSNs

Gjoka

Kurant

Butts

et al. 2010

2010 Proceedings IEEE INFOCOM

547

507

View full text Add to dashboard Cite

show abstract

“…Finally, simple re-weighting is difficult or impossible to apply in the context of many purely data-analytic procedures such as multidimensional scaling or hierarchical clustering. Simulated Importance Resampling [66] provides a useful alternative for RWRW samples, but suffers from well-known problems of asymptotic bias (see [67] for a discussion). This is of less concern for applications such as moment estimation, for which re-weighting is both simple and effective.…”

Section: Findings and Practical Recommendations 1) Choosing Betweementioning

confidence: 99%

Practical Recommendations on Crawling Online Social Networks

Gjoka

Kurant

Butts

et al. 2011

IEEE J. Select. Areas Commun.

222

200

View full text Add to dashboard Cite

Abstract-Our goal in this paper is to develop a practical framework for obtaining a uniform sample of users in an online social network (OSN) by crawling its social graph. Such a sample allows to estimate any user property and some topological properties as well. To this end, first, we consider and compare several candidate crawling techniques. Two approaches that can produce approximately uniform samples are the MetropolisHasting random walk (MHRW) and a re-weighted random walk (RWRW). Both have pros and cons, which we demonstrate through a comparison to each other as well as to the "ground truth." In contrast, using Breadth-First-Search (BFS) or an unadjusted Random Walk (RW) leads to substantially biased results. Second, and in addition to offline performance assessment, we introduce online formal convergence diagnostics to assess sample quality during the data collection process. We show how these diagnostics can be used to effectively determine when a random walk sample is of adequate size and quality. Third, as a case study, we apply the above methods to Facebook and we collect the first, to the best of our knowledge, representative sample of Facebook users. We make it publicly available and employ it to characterize several key properties of Facebook.

show abstract

“…We sample 500 000 sets of parameters using the Latin Hypercube Sampling technique (Iman and Helton, 1988) to conduct the Monte Carlo simulations. We then use the sampling importance resampling (SIR) technique with the observed carbon fluxes at the selected sites to draw the posterior from the prior SAT-TEM simulations (Skare et al, 2003) and collect 50 000 posterior sets of parameters, which are one-tenth of the prior sample size and are suggested to be able to produce stable results (Green et al, 1999;Tang and Zhuang, 2009). We then divide the errors made by the 50 000 sets of parameters into 50 levels (from the highest error level to the lowest) and sampled 50 sets of parameters, one for each level.…”

Section: Model Parameterization and Applicationmentioning

confidence: 99%

Quantification of terrestrial ecosystem carbon dynamics in the conterminous United States combining a process-based biogeochemical model and MODIS and AmeriFlux data

et al. 2011

View full text Add to dashboard Cite

Abstract. Satellite remote sensing provides continuous temporal and spatial information of terrestrial ecosystems. Using these remote sensing data and eddy flux measurements and biogeochemical models, such as the Terrestrial Ecosystem Model (TEM), should provide a more adequate quantification of carbon dynamics of terrestrial ecosystems. Here we use Moderate Resolution Imaging Spectroradiometer (MODIS) Enhanced Vegetation Index (EVI), Land Surface Water Index (LSWI) and carbon flux data of AmeriFlux to conduct such a study. We first modify the gross primary production (GPP) modeling in TEM by incorporating EVI and LSWI to account for the effects of the changes of canopy photosynthetic capacity, phenology and water stress. Second, we parameterize and verify the new version of TEM with eddy flux data. We then apply the model to the conterminous United States over the period 2000-2005 at a 0.05 • × 0.05 • spatial resolution. We find that the new version of TEM made improvement over the previous version and generally captured the expected temporal and spatial patterns of regional carbon dynamics. We estimate that regional GPP is between 7.02 and 7.78 Pg C yr −1 and net primary production (NPP) ranges from 3.81 to 4.38 Pg C yr −1 and net ecosystem production (NEP) varies within 0.08-0. were captured by the model. Our study provides a new independent and more adequate measure of carbon fluxes for the conterminous United States, which will benefit studies of carbon-climate feedback and facilitate policy-making of carbon management and climate.

show abstract

Improved Sampling‐Importance Resampling and Reduced Bias Importance Sampling

Cited by 55 publications

References 36 publications

Walking in Facebook: A Case Study of Unbiased Sampling of OSNs

Walking in Facebook: A Case Study of Unbiased Sampling of OSNs

Practical Recommendations on Crawling Online Social Networks

Quantification of terrestrial ecosystem carbon dynamics in the conterminous United States combining a process-based biogeochemical model and MODIS and AmeriFlux data

Contact Info

Product

Resources

About