While evolving neuro-fuzzy systems have shown promise for learning from non-stationary streaming data with concept drift, most existing models lack transparency due to the limited interpretability of Takagi-Sugeno fuzzy architecture's linear rule consequents. The lack of transparency limits the reliability of crucial applications. To address this limitation, this paper proposes a new evolving neuro-fuzzy system called X-Fuzz that enhances interpretability by integrating the LIME technique to provide local explanations and evaluates them using faithfulness and monotonicity metrics. X-Fuzz is rigorously tested on streaming datasets with diverse concept drifts via prequential analysis. Experiments demonstrate X-Fuzz's capabilities in mining insights from large and dynamic data streams exhibiting diverse concept drifts including abrupt, gradual, recurring contextual, and cyclical drifts. In addition, for online runway exit prediction using real aviation data, X-Fuzz achieved 98.04% accuracy, significantly exceeding recent methods. With its balance of efficiency and transparency, X-Fuzz represents a promising approach for trustworthy evolving artificial intelligence that can handle complex, non-stationary data streams in critical realworld settings. We have made the X-Fuzz source code available in https://github.com/m-ferdaus/X Fuzz for reproducibility and facilitating future research.Impact Statement-The transparency of current evolving neuro-fuzzy systems restricts trustworthiness, limiting adoption in critical real-world applications. X-Fuzz tackles this barrier by transparently learning from non-stationary data via integrated local explanations and rigorous explanation evaluation. In online runway exit prediction, X-Fuzz achieves 98.04% accuracy, exceeding recent models by 6.7%. With its balanced efficiency and transparency, X-Fuzz significantly advances reliable and interpretable evolving AI. By making model behaviors traceable, X-Fuzz expands the safe deployment of adaptive intelligent systems in mission-critical settings across domains. Overall, X-Fuzz enables trustworthy neuro-fuzzy learners that incrementally adapt to dynamic data, overcoming reliability barriers and driving real-world impact.