We use first-principles calculations based on density functional theory to investigate the magnetic exchange interaction of Fe clusters on Rh(111) and Ru(0001). We consider dimers, trimers, tetramers, and pentamers of different shape in fcc and hcp stacking as well as infinite atomic and biatomic chains. From the dimer calculations, we extract the exchange interaction as a function of adatom distance by mapping total energies to a Heisenberg model. The nearest-neighbor (NN) exchange constant is about one order of magnitude smaller than reported for other substrates due to the strong hybridization between the Fe atoms and the partly filled 4d band of the surface. We also find a transition from a ferromagnetic NN exchange interaction for Fe dimers on Rh (111) to an antiferromagnetic one on Ru(0001). The distance-dependent exchange coupling displays a RKKY-like oscillatory behavior, which is nearly inverted for Fe dimers on the Rh(111) surface compared to those on Ru(0001). Unexpectedly, for Fe clusters beyond dimers, a complex trend of the magnetic ground state is observed which alternates between ferro-and antiferromagnetic configurations depending on cluster size and shape. In view of the exchange constants obtained for dimers, it is surprising that on both surfaces small compact clusters are ferromagnetic while open structures such as linear trimers or tetramers become antiferromagnetic. We demonstrate that both vertical and lateral structural relaxations of the clusters are crucial in order to understand this unexpected trend of magnetic order and connected to the competition of direct ferromagnetic exchange among Fe atoms in the cluster and the hybridization with the substrate.