The optimization of the rule base of a fuzzy logic system (FLS) based on evolutionary algorithm has achieved notable results. However, due to the diversity of the deep structure in the hierarchical fuzzy system (HFS) and the correlation of each sub fuzzy system, the uncertainty of the HFS's deep structure increases. For the HFS, a large number of studies mainly use fixed structures, which cannot be selected automatically. To solve this problem, this paper proposes a novel approach for constructing the incremental HFS. During system design, the deep structure and the rule base of the HFS are encoded separately. Subsequently, the deep structure is adaptively mutated based on the fitness value, so as to realize the diversity of deep structures while ensuring reasonable competition among the structures. Finally, the differential evolution (DE) is used to optimize the deep structure of HFS and the parameters of antecedent and consequent simultaneously. The simulation results confirm the effectiveness of the model. Specifically, the root mean square errors in the Laser dataset and Friedman dataset are 0.0395 and 0.0725, respectively with rule counts of rules is 8 and 12, respectively. When compared to alternative methods, the results indicate that the proposed method offers improvements in accuracy and rule counts.