The U.S. Department of Agriculture’s (USDA) Cropland Data Layer (CDL) is a 30 m resolution crop-specific land cover map produced annually to assess crops and cropland area across the conterminous United States. Despite its prominent use and value for monitoring agricultural land use/land cover (LULC), there remains substantial uncertainty surrounding the CDLs’ performance, particularly in applications measuring LULC at national scales, within aggregated classes, or changes across years. To fill this gap, we used state- and land cover class-specific accuracy statistics from the USDA from 2008 to 2016 to comprehensively characterize the performance of the CDL across space and time. We estimated nationwide area-weighted accuracies for the CDL for specific crops as well as for the aggregated classes of cropland and non-cropland. We also derived and reported new metrics of superclass accuracy and within-domain error rates, which help to quantify and differentiate the efficacy of mapping aggregated land use classes (e.g., cropland) among constituent subclasses (i.e., specific crops). We show that aggregate classes embody drastically higher accuracies, such that the CDL correctly identifies cropland from the user's perspective 97% of the time or greater for all years since nationwide coverage began in 2008. We also quantified the mapping biases of specific crops throughout time and used these data to generate independent bias-adjusted crop area estimates, which may complement other USDA survey- and census-based crop statistics. Our overall findings demonstrate that the CDLs provide highly accurate annual measures of crops and cropland areas, and when used appropriately, are an indispensable tool for monitoring changes to agricultural landscapes.