Aerogels are synthetic porous materials derived from sol-gel materials in which the liquid component has been replaced with gas to leave intact solid nanostructures without pore collapse. Recently, aerogels based on natural or synthetic polymers, called polymer or organic aerogels, have been widely explored due to their porous structures and unique properties, such as high specific surface area, low density, low thermal conductivity and dielectric constant. This paper gives a comprehensive review about the most recent progresses in preparation, structures and properties of polymer and their derived carbon-based aerogels, as well as their potential applications in various fields including energy storage, adsorption, thermal insulation and flame retardancy. To facilitate further research and development, the technical challenges are discussed, and several future research directions are also suggested in this review.