In this paper, a stochastic configuration based fuzzy inference system with interpretable fuzzy rules (SCFS-IFRs) is proposed to improve the interpretability and performance of the fuzzy inference system and determine autonomously an appropriate model structure. The proposed SCFS-IFR first accomplishes a fuzzy system through interpretable linguistic fuzzy rules (ILFRs), which endows the system with clear semantic interpretability. Meanwhile, using an incremental learning method based on stochastic configuration, the appropriate architecture of the system is determined by incremental generation of ILFRs under a supervision mechanism. In addition, the particle swarm optimization (PSO) algorithm, an intelligence search technique, is used in the incremental learning process of ILFRs to obtain better random parameters and improve approximation accuracy. The performance of SCFS-IFRs is verified by regression and classification benchmark datasets. Regression experiments show that the proposed SCFS-IFRs perform best on 10 of the 20 data sets, statistically significantly outperforming the other eight state-of-the-art algorithms. Classification experiments show that, compared with the other six fuzzy classifiers, SCFS-IFRs achieve higher classification accuracy and better interpretation with fewer rules.