Non-intrusive and automated detection of pig breeds, particularly from visual standpoint, is important from a food quality tracking perspective, both from the point of view of the vendors as well as the buyers. Colour as well as texture based visual descriptors from muzzle images have been identified, which, serve as breed-identifiers to separate four common pig-breeds: Duroc, Ghungroo, Hampshire and Yorkshire. While these handcrafted visual descriptors by themselves are fairly robust and discriminative, it is recognized that by controlling the decision space either by choosing the feature-type based on colour or texture or by combining features and also selecting the order in which particular breeds are siphoned, classification accuracies can be improved. In that light, is proposed a stable, relatively data-independent, breed-specific, hierarchical tree synthesis and feature selection procedure, based on a breed-pair cluster separation table, aided by some secondary statistics. The proposed approach has been compared with the state of the art Phylogenetic distance based Hierarchical Agglomerative Clustering algorithm (AGNES) and also with the standard decision tree classification algorithm. When completely different sets of pigs were used for training and testing (50-50 split), the proposed algorithm reported relatively high mean classification accuracies of 86.45% for Duroc, 93.02% for Ghungroo, 86.91% for Hampshire and 98.54% for Yorkshire.