Icariside I, the glycosylation product of icaritin, is a novel effective anti-cancer agent with immunological anti-tumor activity. However, very limited natural icariside I content hinders its direct extraction from plants. Therefore, we employed a computer-aided protein design strategy to improve the catalytic efficiency and substrate specificity of the α-L-rhamnosidase from Thermotoga petrophila DSM 13995, to provide a highly-efficient preparation method. Several beneficial mutants were obtained by expanding the active cavity. The catalytic efficiencies of all mutants were improved 16–200-fold compared with the wild-type TpeRha. The double-point mutant DH was the best mutant and showed the highest catalytic efficiency (kcat/KM: 193.52 s−1 M−1) against icariin, which was a 209.76-fold increase compared with the wild-type TpeRha. Besides, the single-point mutant H570A showed higher substrate specificity than that of the wild-type TpeRha in hydrolysis of different substrates. This study provides enzyme design strategies and principles for the hydrolysis of rhamnosyl natural products.