Narrow linewidth pulsed single-frequency fiber laser has the characteristics of excellent coherence and high peak power, and has been widely used in coherent LiDAR, precision measurement and nonlinear frequency conversion. In this work, energy scaling of single-frequency laser with long pulse duration and low repetition rate is demonstrated through leveraging an all-fiber amplifier based on polarization-maintaining (PM) large mode area tapered Yb-doped fiber. With dedicate synchronous pulse pumping for suppressing the inter-pulse ASE effect, and pre-shaping of the seed pulse for compensating the distortion of temporal profile, a 1064 nm pulsed single-frequency laser output with an energy of 370 mJ is realized with a repetition rate of 100 Hz. In addition, the rectangle pulse shape is well maintained with a duration of 1.1 ms. It is believed that the obtained result represents the highest energy ever reported for pulsed single-frequency fiber lasers.