Smallholder agriculture is the main driver of deforestation in the western Amazon, where terrestrial biodiversity reaches its global maximum. Understanding the biodiversity value of the resulting mosaics of cultivated and secondary forest is therefore crucial for conservation planning. However, Amazonian communities are organized across multiple forest types that support distinct species assemblages, and little is known about smallholder impacts across the range of forest types that are essential for sustaining biodiversity. We addressed this issue with a large‐scale field inventory of birds (point counts) and trees (transects) in primary forest and smallholder agriculture in northern Peru across 3 forest types that are key for Amazonian biodiversity. For birds smallholder agriculture supported species richness comparable to primary forest within each forest type, but biotic homogenization across forest types resulted in substantial losses of biodiversity overall. These overall losses are invisible to studies that focus solely on upland (terra firma) forest. For trees biodiversity losses in upland forests dominated the signal across all habitats combined and homogenization across habitats did not exacerbate biodiversity loss. Proximity to forest strongly predicted the persistence of forest‐associated bird and tree species in the smallholder mosaic, and because intact forest is ubiquitous in our study area, our results probably represent a best‐case scenario for biodiversity in Amazonian agriculture. Land‐use planning inside and outside protected areas should recognize that tropical smallholder agriculture has pervasive biodiversity impacts that are not apparent in typical studies that cover a single forest type. The full range of forest types must be surveyed to accurately assess biodiversity losses, and primary forests must be protected to prevent landscape‐scale biodiversity loss.