In this work, we employed gain-managed nonlinear amplification (GMNA) technology to achieve a pulse characterized by a smooth spectral profile and an exceptionally wide spectral bandwidth. Then this pulse was injected into a chirped pulse amplification (CPA) system for amplification, and a high energy, narrow pulse duration fiber laser system was successfully built. The initial seed source for system was a self-made mode-locked fiber oscillator that utilized a nonlinear amplifying loop mirror (NALM). This oscillator produced ultrashort pulses with a pulse duration of 10.96 ps at a repetition rate of 11.52 MHz. The spectral width of the mode-locked oscillator was significantly broadened from 5.83 nm to 63.97 nm using GMNA technology. Furthermore, the spectral profile, which initially exhibited severe oscillation structures, was reshaped into a smooth profile through the application of GMNA. Subsequently, the pulse energy was increased through CPA amplification. Finally, in the case of a central wavelength of 1064 nm and a repetition rate of 500 kHz, a pulse with an average power of 20.02 W, a single pulse energy of 40 μJ, a pulse duration of 179 fs, and a peak power of 224 MW was obtained. This fiber laser system has great prospect of application in clinical medicine and precision manufacturing due to its high energy and ultra-short pulse duration.