Passive particle tracking of diffusive paths in soft matter, coupled with analysis of the path data, is firmly established as a fundamental methodology for characterization of both diffusive transport properties (the focus here) and linear viscoelasticity. For either focus, particle time series are typically analyzed by ensemble averaging over paths, a perfectly natural protocol for homogeneous materials or for applications where mean properties are sufficient. Many biological materials, however, are heterogeneous over length scales above the probe diameter, and the implications of heterogeneity for biologically relevant transport properties (e.g. diffusive passage times through a complex fluid layer) motivate this paper. Our goals are three-fold: first, to detect heterogeneity as reflected by the ensemble path data; second, to further decompose the ensemble of particle paths into statistically distinct clusters; and third, to fit the path data in each cluster to a model for the underlying stochastic process. After reviewing current best practices for detection and assessment of heterogeneity in diffusive processes, we introduce our strategy toward the first two goals with methods from the statistics and machine learning literature that have not found application thus far to passive particle tracking data. We apply an analysis based solely on the path data that detects heterogeneity and yields a decomposition of particle paths into statistically distinct clusters. After these two goals are achieved, one can then pursue model-fitting. We illustrate these heterogeneity metrics on diverse datasets: for numerically generated and experimental particle paths, with tunable and unknown heterogeneity, on numerical models for simple diffusion and anomalous sub-diffusion, and experimentally on sucrose, hyaluronic acid, agarose, and human lung culture mucus solutions.