Many countries and regions around the world are experiencing this development dilemma, and Northeast China is a typical representative. To explore the reason for the backwardness of Northeast China, we analyze the formation mechanism and efficiency of enterprise agglomeration in this research. Based on the panel data of 34 prefectural level or above cities in Northeast China and 241 citescities in other regions from 1999 to 2015, a fixed effects panel model is used, considering enterprise agglomeration, foreign direct investment (FDI), investment in fixed assets, and the non-agricultural industry structure. It is found that enterprise agglomeration has a highly significant negative impact on urban productivity in Northeast China, and this phenomenon of agglomeration diseconomy in the northeast is most significant among the four regions of the country. Furthermore, according to the moderation mechanism analysis, the enterprise agglomeration in Northeast China reduces the pulling effects of fixed asset investment and FDI on economic efficiency but enhances the promotion effect of the non-agricultural industry structure on urban productivity. After the robustness test and extension analysis, this study determines that the enterprise agglomeration in Northeast China does not effectively stimulate the driving role of investments. Finally, we discuss measures that can help resolve the current agglomeration diseconomy problem in Northeast China and achieve sustainable development.