In Vietnam, agriculture is a key sector that promotes economic growth and poverty reduction. Therefore, improving productivity in agriculture is indispensable to the sustainability of the country. This research examined productivity and its determinants from 420 enterprises operating in agriculture. Productivity was measured as the total factor productivity (TFP) obtained from fixed and random effects models. The determinants of TFP including size and age, share of state and foreign ownership, export, accessibility to Internet and bank loan of firms, controlled for year fixed effects, were analyzed. It was shown that 74.6% companies in the agricultural sector were small in size (<10<200 employees). Although the number of large firms (>300 employees) explained 10.6%, they had a remarkable and positive TFP (38.8%, p < 0.01), while both small and very small (<10, and <200 employees, respectively) had strikingly negative TFP values (−71.3% and −32.1%, respectively, p < 0.01), as compared to the medium sizes (<200<300 employees). It was also revealed that although foreign ownership was only 3.8% on average, it had a notably positive effect on TFP (55.0%, p < 0.01). In contrast, state ownership accounted for 30.7%, but it had a negative influence on TFP (−7.5%). The export contributed a negligible and statistically significant effect to TFP (2.6%), which might be attributed to a limited number of firms (4.5%) having mobility in agricultural export. 73% received a bank loan, and only 18.2% had access to the Internet, but both of them yielded remarkable TFP values (18.5%, p < 0.01 and 3.4%, p < 0.05 respectively). The Hausman test indicated that the fixed effects (FE) model was more effective than the random effects (RE) model to estimate the TFP. The findings of this study suggested that reform efforts should focus on improving the productivity of small agricultural enterprises. In addition, foreign investment, effective use of bank loan and Internet accessibility should be further enhanced. The results of this study may provide insights for policymakers who aim to improve the productivity in agricultural enterprises and thereby contribute to the sustainable growth of the country.