Globally, the increase in human population continues to threaten the sustainability of agricultural systems. Despite the fast-growing population in Sub-Saharan Africa (SSA) and the efforts in improving the productivity of crops, the increase in the yield of crops per unit area is still not promising. The productivity of crops is primarily constrained by inadequate levels of soil nutrients to support optimum crop growth and development. However, smallholder farmers occasionally use fertilizers, and the amount applied is usually small and does not meet plant requirements. This is due to the unaffordability of the cost of fertilizers, which is enough to suffice the crop requirement. Therefore, there is a need for alternative affordable and effective fertilization methods for sustainable intensification and improvement of the smallholder farming system's productivity. This study was designed to evaluate the symbiotic performance of indigenous soybean nodulating rhizobia in selected agricultural soils of Tanzania. In total, 217 rhizobia isolates were obtained from three agroecological zones, i.e., eastern, northern, and southern highlands. The isolates collected were screened for N2 fixing abilities under in vitro (nitrogen-free medium) and screen house conditions. The results showed varying capabilities of isolates in nitrogen-fixing both under in vitro and screen house conditions. Under in vitro experiment, 22% of soybean rhizobia isolates were identified to have a nitrogen-fixing capability on an N-free medium, with the highest N2-fixing diameter of 1.87 cm. In the screen house pot experiment, results showed that soybean rhizobia isolate significantly (P < 0.001) influenced different plant growth and yield components, where the average shoot dry weight ranged from 2.49 to 10.98 g, shoot length from 41 to 125.27 cm whilst the number of leaves per plant ranged from 20 to 66. Furthermore, rhizobia isolates significantly (P = 0.038) increased root dry weight from 0.574 to 2.17 g. In the case of symbiotic parameters per plant, the number of nodules was in the range of 0.33–22, nodules dry weight (0.001–0.137 g), shoot nitrogen (2.37–4.97%), total nitrogen (53.59–6.72 g), and fixed nitrogen (46.878–0.15 g) per plant. In addition, the results indicated that 51.39% of the tested bacterial isolates in this study were ranked as highly effective in symbiosis, suggesting that they are promising as potential alternative biofertilizers for soybean production in agricultural soils of Tanzania to increase productivity per unit area while reducing production cost.