Cutting down high dimensional data with Fuzzy weighted forests (FWF)

Abstract

Takagi-Sugeno-Kang (TSK) rule-based fuzzy systems struggle to deal with high dimensional data and suffer from the curse of dimensionality. As the number of input features increases, the number of rules increases exponentially, which reduces the model's interpretability rapidly. This paper presents a novel fuzzy weighted forest aggregation method to effectively model high dimensional data by reducing the number of fuzzy rules, without sacrificing accuracy. The fuzzy weighted forest is comprised of several fuzzy weighted trees. Each tree is created based on a subset of features captured across different parts of the input space. Given 𝒏 input features and 𝑵 samples, every fuzzy tree is randomly assigned 𝒏 ′ features and 𝑵 ′ samples, where 𝒏 ′ and 𝑵 ′ are significantly smaller than 𝒏 and 𝑵 respectively. Each path within a tree, from root to leaf, forms a fuzzy rule. The non-leaf nodes represent the antecedents of rules, and the leaf node represents the consequents. This study shows how the proposed method utilizes pruning to significantly reduce the number of fuzzy rules. This method therefore creates a less complex model whilst achieving high accuracy comparable with, and sometimes better than, existing state-ofthe-art TSK-based fuzzy models.