A Robust and High-Dimensional Clustering Algorithm Based on Feature Weight and Entropy

Du, Xinzhi

doi:10.3390/e25030510

Cited by 3 publications

(2 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A generalization of the objective function is shown in [35]. Other changes in the objective function are oriented to assign importance weights or weighting factors, for example, to the Euclidean distance [36,37], objects and clusters [30,38,39], and attributes [30,40]. These variants include as the convergence indicator the difference of the objective function values in two consecutive iterations, which is denoted by ∆ old J m ; see Expression (10).…”

Section: Improvements Of the Fcm Algorithm That Modify The Objective ...mentioning

confidence: 99%

A New Criterion for Improving Convergence of Fuzzy C-Means Clustering

Pérez-Ortega,

Moreno-Calderón,

Roblero-Aguilar

et al. 2024

Axioms

View full text Add to dashboard Cite

One of the most used algorithms to solve the fuzzy clustering problem is Fuzzy C-Means; however, one of its main limitations is its high computational complexity. It is known that the efficiency of an algorithm depends, among other factors, on the strategies for its initialization and convergence. In this research, a new convergence strategy is proposed, which is based on the difference of the objective function values, in two consecutive iterations, expressed as a percentage of its value in the next to the last one. Additionally, a new method is proposed to optimize the selection of values of the convergence or stop threshold of the algorithm, which is based on the Pareto principle. To validate our approach, a collection of real datasets was solved, and a significant reduction in the number of iterations was observed, without affecting significantly the solution quality. Based on the proposed method and the experiments carried out, we found it is convenient to use threshold values equal to 0.73 and 0.35 if a decrease in the number of iterations of approximately 75.2% and 64.56%, respectively, is wanted, at the expense of a reduction in solution quality of 2% and 1%, respectively. It is worth mentioning that, as the size of the datasets is increased, the proposed approach tends to obtain better results, and therefore, its use is suggested for datasets found in Big Data and Data Science.

show abstract

Section: Improvements Of the Fcm Algorithm That Modify The Objective ...mentioning

confidence: 99%

A New Criterion for Improving Convergence of Fuzzy C-Means Clustering

Pérez-Ortega,

Moreno-Calderón,

Roblero-Aguilar

et al. 2024

Axioms

View full text Add to dashboard Cite

show abstract

“…However, their training approach involved a possibility of false negative sampling (Huynh et al, 2022), where the negative pair's distant tile (taken from a random location on a different day) does not necessarily guarantee a dissimilarity in their cloud system's structure and distribution. Further, employing high-dimensional features in HC has performance and scalability issues (Du, 2023;Gilpin et al, 2013). Janssens et al (2021) assumes a linear combination of traditional cloud metrics for describing the cloud systems.…”

Section: 1029/2024gl108889mentioning

confidence: 99%

Capturing the Diversity of Mesoscale Trade Wind Cumuli Using Complementary Approaches From Self‐Supervised Deep Learning

Chatterjee,

Schnitt,

Bigalke

et al. 2024

Geophysical Research Letters

View full text Add to dashboard Cite

At mesoscale, trade wind clouds organize with various spatial arrangements, shaping their effect on Earth's energy budget. Representing their fine‐scale dynamics even at 1 km scale climate simulations remains challenging. However, geostationary satellites (GS) offer high‐resolution cloud observation for gaining insights into trade wind cumuli from long‐term records. To capture the observed organizational variability, this work proposes an integrated framework using a continuous followed by discrete self‐supervised deep learning approach, which exploits cloud optical depth from GS measurements. We aim to simplify the entire mesoscale cloud spectrum by reducing the image complexity in the feature space and meaningfully partitioning it into seven classes whose connection to environmental conditions is illustrated with reanalysis data. Our framework facilitates comparing human‐labeled mesoscale classes with machine‐identified ones, addressing uncertainties in both methods. It advances previous methods by exploring transitions between regimes, a challenge for physical simulations, and illustrates a case study of sugar‐to‐flower transitions.

show abstract

A novel high-dimensional clustering algorithm and its application in financial data forecasting

Qian,

Du,

et al. 2024

Third International Conference on Electronic Information Engineering, Big Data, and Computer Technology (EIBDCT 2024)

View full text Add to dashboard Cite

The purpose of this paper is to study the nonlinear characteristics of stock market price rise and fall, and optimize the characteristics of high-dimensional data for the challenges brought by the sparsity and complexity of high-dimensional data to the traditional clustering algorithms. Explore the impact of the multidimensional characteristics and sparsity of stock data on clustering analysis, and propose a high-dimensional clustering algorithm based on Pareto-optimal Steiner points. Traditional clustering algorithms are deficient in dealing with nonlinear data and spatial clustering, while randomly selected clustering centers are easily interfered by market noise, which affects the accuracy of clustering results. In this paper, more accurate clustering results are achieved by selecting Steiner points with Pareto optimality as clustering centers. And the algorithm is applied to the analysis of the stock market, which can effectively help investors choose the appropriate investment portfolio and has important practical application value. Meanwhile, the distance matrix is constructed as the basis of clustering analysis through the preprocessing of stock data and distance metric. The innovation of this paper is that the Steiner points can better represent the center of the clusters thus effectively reducing the computational complexity and storage overhead and improving the efficiency of the algorithm. And the principle of Pareto optimality is applied to the high-dimensional clustering algorithm, which improves the accuracy of stock market analysis.

show abstract

A Robust and High-Dimensional Clustering Algorithm Based on Feature Weight and Entropy

Cited by 3 publications

References 35 publications

A New Criterion for Improving Convergence of Fuzzy C-Means Clustering

A New Criterion for Improving Convergence of Fuzzy C-Means Clustering

Capturing the Diversity of Mesoscale Trade Wind Cumuli Using Complementary Approaches From Self‐Supervised Deep Learning

A novel high-dimensional clustering algorithm and its application in financial data forecasting

Contact Info

Product

Resources

About