By consuming ammonium and nitrite, anammox bacteria form an important functional guild in nitrogen cycling in many environments including marine sediments. Recent studies have shown that anammox bacteria can consume most of the upwardly diffusing ammonium from deep anoxic sediments; however, their impact on the other important substrate nitrite has not been well characterized. Here we show niche partitioning of two anammox families emerges in a 2.4 m long mostly anoxic sediment core retrieved from the Nordic Seas. We document high abundances of anammox bacteria in most investigated sediment layers, with two distinct anammox abundance maxima in two nitrite consumption zones. Between the two anammox abundance maxima, nitrite accumulates as observed in other marine sediment sites and aquatic environments, indicating anammox bacteria play a fundamental role in modulating the nitrite distribution. Anammox bacteria in the upper nitrite consumption zone are dominated by the Candidatus Bathyanammoxibiaceae family, while Ca. Scalinduaceae family dominate at the lower zone. A high-quality representative Ca. Bathyanammoxibiaceae genome is recovered, which, comparing to Ca. Scalindua sediminis, the representative of Scalinduaceae in marine sediments, has fewer high affinity ammonium transporters and lacks the capacity to access alternative substrates or energy sources such as urea and cyanate. These features may restrict Ca. Bathyanammoxibiaceae to conditions of higher ammonium concentrations or fluxes, and therefore drive the observed niche partitioning. These findings improve our understanding about nitrogen cycling in marine sediments by revealing the association between nitrite accumulation and niche partitioning of anammox bacteria.