In this paper, a detailed mathematical analysis of the optimality of the premise and consequent parts of the recently introduced first-order Autonomous Learning Multi-Model (ALMMo) neuro-fuzzy system is conducted. A novel self-boosting algorithm for structure-and parameter-optimization is, then, introduced to the ALMMo, which results in the self-boosting ALMMo (SBALMMo) neuro-fuzzy system. By minimizing the objective functions with the previously collected data, the SBALMMo is able to optimize its system structure and parameters in few iterations. Numerical examples based benchmark datasets and real-world problems demonstrate the effectiveness and validity of the SBALMMo, and show the strong potential of the proposed approach for real applications.