The existing literature on smart city pilots mainly focuses on the city level and rarely addresses the firm level. This paper assesses the impact of smart city pilot policy (SCP) on firms’ total factor productivity (TFP) and explores the impact of SCP under different heterogeneities as well as the mechanisms of action of the SCP. The LP approach is used in this paper to measure firms’ TFP, and the impact of SCP is analyzed by the DID model with firms’ panel data from 2009 to 2019 as research objects. First, it was found that the SCP can significantly increase the TFP of firms (0.041). Second, through heterogeneity analysis, we found that SCP can strengthen the monopoly position of monopolistic firms and state-owned enterprises. Moreover, the SCP can also alleviate the development imbalance of TFP between firms in coastal and non-coastal areas. In addition, SCP can significantly improve TFP of heavy polluting enterprises. Finally, we find that the important ways for SCP to improve firms’ TFP is increasing investment in technological innovation, talent agglomeration, attracting financing, improving resource allocation efficiency, and digital transformation. The study provides unique insights for policy makers and business managers in China and other emerging countries to enhance TFP and achieve corporate sustainable development.