Comparison of Similarity Measures for Categorical Data in Hierarchical Clustering

Šulc, Zdeněk; Řezanková, Hana

doi:10.1007/s00357-019-09317-5

Cited by 37 publications

(21 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Goodall and Eskin aim to weight values higher that match infrequently, while IOF and Lin give greater weight to values that match frequently, and lower weight to infrequent matches ( Boriah, Chandola & Kumar, 2008 ). All of these measures have been shown previously to perform well on different datasets in different conditions ( Šulc, 2016 ), emphasising the need to test a range of methodologies when clustering ecological data.…”

Section: Discussionmentioning

confidence: 99%

“…To select the best distance matrix and clustering method for our data we utilised internal evaluation measures available from nomclust ( Šulc & Řezankovà, 2015 ). The within-cluster entropy coefficient (WCE) is a measure of compactness which evaluates the variability of each cluster by calculating a measure of normalised entropy (the number of variables that have the same categories from each of the variables evaluated) ( Šulc, 2016 ). WCE is measured from 0 to 1, where a lower value indicates intra-cluster homogeneity.…”

Section: Methodsmentioning

confidence: 99%

“…To select the number of groups we use the pseudo F coefficient based on the entropy (PSFE), a measure of separation ( Šulc, 2016 ). The PSFE is a measure of entropy of the between- and within-cluster variability adjusted for the number of clusters and number of objects in the cluster where a higher value indicates a better grouping: where n is the number of observations and k is the number of clusters, nWCE (1) is the variability in the whole dataset, and nWCE ( k ) the within-cluster variability in the k -cluster solution.…”

Section: Methodsmentioning

confidence: 99%

“…Even with dozens of cluster evaluation metrics available, they are rarely utilised as there is no single cluster evaluation index that can outperform others ( Arbelaitz et al, 2013 ; Milligan & Cooper, 1985 ). There are two possible ways of evaluating the distribution of variables to clusters—internal and external ( Šulc, 2016 ). External indices are used to evaluate supervised learning problems where the model solution is evaluated against the known solution.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Creating functional groups of marine fish from categorical traits

Ladds

Sibanda

Arnold

et al. 2018

PeerJ

View full text Add to dashboard Cite

BackgroundFunctional groups serve two important functions in ecology: they allow for simplification of ecosystem models and can aid in understanding diversity. Despite their important applications, there has not been a universally accepted method of how to define them. A common approach is to cluster species on a set of traits, validated through visual confirmation of resulting groups based primarily on expert opinion. The goal of this research is to determine a suitable procedure for creating and evaluating functional groups that arise from clustering nominal traits.MethodsTo do so, we produced a species by trait matrix of 22 traits from 116 fish species from Tasman Bay and Golden Bay, New Zealand. Data collected from photographs and published literature were predominantly nominal, and a small number of continuous traits were discretized. Some data were missing, so the benefit of imputing data was assessed using four approaches on data with known missing values. Hierarchical clustering is utilised to search for underlying data structure in the data that may represent functional groups. Within this clustering paradigm there are a number of distance matrices and linkage methods available, several combinations of which we test. The resulting clusters are evaluated using internal metrics developed specifically for nominal clustering. This revealed the choice of number of clusters, distance matrix and linkage method greatly affected the overall within- and between- cluster variability. We visualise the clustering in two dimensions and the stability of clusters is assessed through bootstrapping.ResultsMissing data imputation showed up to 90% accuracy using polytomous imputation, so was used to impute the real missing data. A division of the species information into three functional groups was the most separated, compact and stable result. Increasing the number of clusters increased the inconsistency of group membership, and selection of the appropriate distance matrix and linkage method improved the fit.DiscussionWe show that the commonly used methodologies used for the creation of functional groups are fraught with subjectivity, ultimately causing significant variation in the composition of resulting groups. Depending on the research goal dictates the appropriate strategy for selecting number of groups, distance matrix and clustering algorithm combination.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Creating functional groups of marine fish from categorical traits

Ladds

Sibanda

Arnold

et al. 2018

PeerJ

View full text Add to dashboard Cite

show abstract

“…Abstract norms (SN), this variable is measured as follows: Respondents were asked who they consider responsible for energy conservation?Four baseline groups were considered to answer this question: "my friends", "government", "my family members" and "my neighbors" (Cronbach's alpha: 0.73, my average friends: 2.46 (standard deviation: 1.05), Average government: 1.72 (standard deviation: 0.9), average family members: 2.09 (standard deviation: 1.01), average neighbors: 2.24 (standard deviation: 1.07)). On average, households blamed all four groups, but the government, with the lowest average, is completely in agreement with the options, and this could indicate that people in Iranian society still do not want or can not believe that The people are ahead of the government in pursuing the right pattern of public consumption(see Table 3) [97][98][99][100][101][102][103][104][105]. Understanding Behavior Control (PBC) goes back to whether respondents feel that they can save electricity in their homes.…”

Section: Variables Of the Theory Of Planned Behaviormentioning

confidence: 99%

Behavioral Economics and Energy Consumption: Behavioral Data Analysis the Role of Attitudes and Beliefs on Household Electricity Consumption in Iran

Norouzi

Fani

Bahramani

et al. 2021

JAIBD

View full text Add to dashboard Cite

The average electricity consumption in Iranian households is higher than the world average. This can be due to price factors (such as cheap electricity in the country) and non-price factors (such as socio-demographic variables and psychological factors). In this study, non-price factors such as socio-demographic variables and psychological factors in the electricity consumption of urban households in Tehran were investigated. In this regard, using the theoretical foundations of behavioral economics and the psychology of planned behavior, this issue was analyzed. This study collected information on household electricity consumption behavior through a questionnaire and fieldwork from 2560 Tehran households. Results Using econometric techniques, linear regression was estimated, the dependent variable of which was electricity consumption (45 days in winter 2019) and its independent variables including socio-demographic variables (age, sex, number of household members, income) and The variables of the theory of planned behavior (attitude, mental norms and perceived behavioral control) showed that income and the number of household members have a significant and positive effect on electricity consumption, but gender has no significant effect. Of the psychological variables, only perceived behavioral control has a significant effect on electricity consumption. These results show that the consumer does not have a positive attitude towards saving, and mental and social norms do not encourage him to reduce electricity consumption, and they are not effective in consumption control. Finally, the study results were analyzed using behavioral biases that may cause attitudes and beliefs not to lead to action.

show abstract

Genomic selection performs as effectively as phenotypic selection for increasing seed yield in soybean

et al. 2022

View full text Add to dashboard Cite

Increasing the rate of genetic gain for seed yield remains the primary breeding objective in both public and private soybean [Glycine max (L.) Merr.] breeding programs. Genomic selection (GS) has the potential to accelerate the rate of genetic gain for soybean seed yield. Limited studies to date have validated GS accuracy and directly compared GS with phenotypic selection (PS), and none have been reported in soybean. This study conducted the first empirical validation of GS for increasing seed yield using over 1,500 lines and over 7 yr (2010)(2011)(2012)(2013)(2014)(2015)(2016) of replicated experiments in the University of Nebraska-Lincoln soybean breeding program. The study was designed to capture the varying genetic relatedness of the training population to three validation sets: two large biparental populations (TBP-1 and TBP-2) and a large validation set comprised of 457 preselected advanced lines derived from 45 biparental populations (TMP). We found that prediction accuracy (.54) realized in our validation experiments was comparable with what we obtained from a series of cross-validation experiments (.64). Both GS and PS were more effective for increasing the population mean performance compared with random selection (RS). We found a selection advantage of GS over PS, where higher genetic gain and identification of top-performing lines was maximized at 10-20% selected proportion. Genomic selection led to small increases in genetic similarity when compared with PS and RS presumably because of a significant shift on allelic frequencies toward the extremes, suggesting that it could erode genetic diversity more quickly. Overall, we found that GS can perform as effectively as PS but that measures should be considered to protect against loss of genetic variance when using GS.

show abstract

Comparison of Similarity Measures for Categorical Data in Hierarchical Clustering

Cited by 37 publications

References 21 publications

Creating functional groups of marine fish from categorical traits

Creating functional groups of marine fish from categorical traits

Behavioral Economics and Energy Consumption: Behavioral Data Analysis the Role of Attitudes and Beliefs on Household Electricity Consumption in Iran

Genomic selection performs as effectively as phenotypic selection for increasing seed yield in soybean

Contact Info

Product

Resources

About