Hybrid single linkage clustering

Traditional clustering methods, whether hierarchical or non-hierarchical, are fundamental techniques in multivariate statistics that are widely used in data analysis. Nonetheless, these approaches have drawn significant criticism, particularly regarding their limitations when applied to large or complex datasets. Hierarchical clustering methods, though intuitive, lack a clear objective function and are inflexible, potentially leading to suboptimal solutions. Non-hierarchical methods are typically computationally efficient, but highly sensitive to initial conditions and frequently require pre-specification of the number of clusters, which is often unknown. To try to overcome some of these limitations, a novel hybrid method that combines hierarchical and nonhierarchical approaches is proposed. It is named Hybrid Single linkage, briefly HSL, because the methodology resembles the single linkage hierarchical clustering, trying to solve the well-known limitation of the chaining effect. The HSL methodology begins by partitioning the data using a new non-hierarchical algorithm that maximizes the between-cluster distance, based on the single linkage criterion, which defines the distance between clusters as the minimum distance between points in different groups. This step results in an initial partition that tries to identify optimally separated clusters. The method then constructs a hierarchical structure, continually maximizing the inter-cluster distance at each level to ensure consistency in the objective function throughout the process. A key advantage of HSL is the ability to identify an optimal partition and a parsimonious dendrogram around this partition, optimizing an objective function. This flexibility makes HSL adaptive to different data analyses. Simulated data sets together with benchmarks are used to show the features of HSL.