Microbial heterotrophic activity is a major process regulating the flux of dissolved organic matter (DOM) in the ocean, while the characteristics of this DOM strongly influence its microbial utilization and fate in the ocean. In order to broaden the vertical resolution of leucine-to-carbon conversion factors (CFs), needed for converting substrate incorporation into biomass production by heterotrophic bacteria, 20 dilution experiments were performed in the North Atlantic Ocean. We found a depth-stratification in empirical CFs values from epipelagic to bathypelagic waters (4.00 ± 1.09 to 0.10 ± 0.00 kg C mol Leu−1). Our results demonstrated that the customarily used theoretical CF of 1.55 kg C mol Leu−1 in oceanic samples can lead to an underestimation of prokaryotic heterotrophic production in epi- and mesopelagic waters, while it can overestimate it in the bathypelagic ocean. Pearson correlations showed that CFs were related not only to hydrographic variables such as temperature, but also to specific phylogenetic groups and DOM quality and quantity indices. Furthermore, a multiple linear regression model predicting CFs from relatively simple hydrographic and optical spectroscopic measurements was attempted. Taken together, our results suggest that differences in CFs throughout the water column are significantly connected to DOM, and also reflect differences linked to specific prokaryotic groups.