Evaluating the outcomes and tracking the trajectory of biodiversity offsets is essential to demonstrating their effectiveness as a mechanism to conciliate development and conservation. We reviewed the literature to determine the principles that should underpin biodiversity offset planning and the criteria for offset evaluation at the project level. According to the literature, the core principles of equivalence, additionality, and permanence are used as criteria to evaluate conservation outcomes of offsets. We applied the criteria to evaluate offsets of a large iron ore mining project in the Atlantic Forest in Brazil. We examined equivalence in terms of the amount of area per biodiversity value affected and fauna and flora similarity, additionality in terms of landscape connectivity, and permanence in terms of guarantees to ensure protection and restoration offsets lasting outcomes. We found an offset ratio (amount of affected area:offset area) of 1:1.8 for forests and 1:2 for grasslands. Ecological equivalence (i.e., similarity between affected and offset areas) was found for forested areas, but not for ferruginous rupestrian grasslands or for fauna. Landscape metrics showed that connectivity improved relative to the preproject situation as a result of locating restoration offsets in the largest and best‐connected forest patch. Permanence of offsets was addressed by establishing covenants and management measures, but financial guarantees to cover maintenance costs after mine closure were lacking. Offsets should be equivalent in type and size, provide conservation outcomes that would not be obtained without them (additionality), and be lasting (permanence). To monitor and evaluate offsets, it is necessary to determine how well these 3 principles are applied in the planning, implementation, and maintenance of offsets. Achieving measurable conservation outcomes from offsets is a long‐term endeavor that requires sustained management support, and is information intensive. Thus, offsets require ongoing monitoring and evaluation as well as adaptive management.