Arthropods are critical ecosystem components due to their high diversity and sensitivity to perturbation. Furthermore, due to their ease of capture they are often the focus of environmental health surveys. There is much debate regarding the best sampling method to use in these surveys. Sweep netting and pan trapping are two sampling methods commonly used in agricultural arthropod surveys, but have not been contrasted in natural grassland systems at the community level. The purpose of this study was to determine whether sweep netting was effective at estimating arthropod diversity at the community level in grasslands or if supplemental pan trapping was needed. Arthropods were collected from grassland sites in Montana, USA, in the summer of 2011. The following three standardized evaluation criteria (consistency, reliability, and precision) were developed to assess the efficacy of sweep netting and pan trapping, based on analyses of variations in arthropod abundances, species richness, evenness, capture frequency, and community composition. Neither sampling method was sufficient in any criteria to be used alone for community-level arthropod surveys. On a taxa-specific basis, however, sweep netting was consistent, reliable, and precise for Thysanoptera, infrequently collected (i.e., rare) insects, and Arachnida, whereas pan trapping was consistent, reliable, and precise for Collembola and bees, which is especially significant given current threats to the latter's populations worldwide. Species-level identifications increase the detected dissimilarity between sweep netting and pan trapping. We recommend that community-level arthropod surveys use both sampling methods concurrently, at least in grasslands, but likely in most nonagricultural systems. Target surveys, such as monitoring bee communities in fragmented grassland habitat or where detailed information on behavior of the target arthropod groups is available can in some instances employ singular methods. As a general ecological principle, consistency, reliability, and precision are appropriate criteria to evaluate the applicability of a given sampling method for both community-level and taxa-specific arthropod surveys in any ecosystem.