Future wireless communication systems are facing with many challenges due to their complexity and diversification. Orthogonal frequency division multiplexing (OFDM) in 4G cannot meet the requirements in future scenarios, thus alternative multicarrier modulation (MCM) candidates for future physical layer have been extensively studied in the academic field, for example, filter bank multicarrier (FBMC), generalized frequency division multiplexing (GFDM), universal filtered multicarrier (UFMC), filtered OFDM (F-OFDM), and so forth, wherein the prototype filter design is an essential component based on which the synthesis and analysis filters are derived. This paper presents a comprehensive survey on the recent advances of finite impulse response (FIR) filter design methods in MCM based communication systems. Firstly, the fundamental aspects are examined, including the introduction of existing waveform candidates and the principle of FIR filter design. Then the methods of FIR filter design are summarized in details and we focus on the following three categories—frequency sampling methods, windowing based methods and optimization based methods. Finally, the performances of various FIR design methods are evaluated and quantified by power spectral density (PSD) and bit error rate (BER), and different MCM schemes as well as their potential prototype filters are discussed.