Biomass materials with representative morphologies and compositions were employed to study the activation effect of KHCO3. As the activation time increased from 1 to 3 h, the products derived from puffed rice and pleurotus eryngii achieved a hierarchical porous structure, while the products derived from cotton still presented a microporous structure. In the electrochemical test of a three-electrode system, the specific capacitance of these products was 352, 319, and 216 F g−1, respectively. In the two-electrode system, the PR-2-based symmetric supercapacitor presented with a specific capacitance of 280.7 F g−1 at 0.5 A g−1, and the energy density of 14.03 Wh kg−1 at 150.04 W kg−1 and an energy density retention of 73.7% was at an even higher power density of 8380.4 W kg−1. After 10,000 cycles of charging and discharging at 5 A g−1, the specific capacitance retention of the supercapacitor reached 108.8%. Based on the experimental analysis, a likely mechanism for the formation of pores was proposed. The results indicate that biomass materials with soft layered or a network structure are the best candidates to obtain a hierarchical porous structure by KHCO3 activation.