A Cluster Approach to Cloud Cover Classification over South America and Adjacent Oceans Using a k-means/k-means++ Unsupervised Algorithm on GOES IR Imagery

Abstract

An unsupervised k-means/k-means++ clustering algorithm was implemented on daily images of standardized anomalies of brightness temperature (Tb) derived from the Geostationary Operational Environmental Satellite (GOES)-13 infrared data for the period 1 December 2010 to 30 November 2016. The goal was to decompose each individual Tb image into four clusters that captures the characteristics of different cloud regimes. The extracted clusters were ordered by their mean value in an ascending fashion so that the lower the cluster order, the higher the clouds they represent. A linear regression between temperature and height with temperature used as the predictor was conducted to estimate cloud top heights (CTHs) from the Tb values. The analysis of the results was performed in two different ways: sample dates and seasonal features. Cluster 1 is the less dominant one, representing clouds with the highest tops and variabilities. Cluster 4 is the most dominant one and represents a cloud regime that spans the lowest 2 km of the troposphere. Clusters 2 and 3 are entangled in the sense that both have their CTHs spanning the middle troposphere. Correlations between the monthly time series of the number of pixels in each cluster and of the entropy with several circulation indices are also introduced. Additionally, a fractal-related analysis was carried out on cluster 1 in order to resolve cirrus and cumulonimbus.

A novel heuristic method for detecting overfit in unsupervised classification of climate model data

“…• Akaike information criterion (AIC; Akaike, 1973), used in Tandeo et al (2014) and Sonnewald et al (2019). • Caliński and Harabasz (1974), used in Yuchechen et al (2020).…”

“…Unsupervised classification methods, that is, methods that do not know a priori what the properties of these groups might be, have proven adept at identifying coherent spatial structures within climate data, even when no spatial information is supplied to the algorithm. In studies of ocean and atmospheric data, two commonly used unsupervised classification methods are k-means (Solidoro et al, 2007; Hjelmervik and Hjelmervik, 2013; 2014; Hjelmervik et al, 2015; Sonnewald et al, 2019; Houghton and Wilson, 2020; Yuchechen et al, 2020; Liu et al, 2021) and Gaussian mixture modeling (GMM) (Hannachi and O’Neill, 2001; Hannachi, 2007; Tandeo et al, 2014; Maze et al, 2017a; Jones et al, 2019; Crawford, 2020; Sugiura, 2021; Zhao et al, 2021; Fahrin et al, 2022). K-means attempts to find coherent groups by “cutting” the abstract feature space using hyperplanes, whereas GMM attempts to represent the underlying covariance structure in abstract feature space using a linear combination of multi-dimensional Gaussian functions.…”

“…Other common statistical methods used to compare clustering results by class number include the following (a nonexhaustive list): Akaike information criterion (AIC; Akaike, 1973), used in Tandeo et al (2014) and Sonnewald et al (2019). Caliński and Harabasz (1974), used in Yuchechen et al (2020). Davies–Bouldin index (DBI; Davies and Bouldin, 1979), used in Solidoro et al (2007), Silhouette coefficient (Rousseeuw, 1987), used in Houghton and Wilson (2020), criteria based on confidence intervals, used in Hannachi (2007).…”