H.._.ierarchical S. cene Structures (HSS) can be used to represent the perceptual structures found in segmented images which arise from inter-region relationships between the image segments. These relationships include continuation of boundaries, proximity, enclosure, and similarity of shape, intensity, size, and directionality. Our prototype system for HSS creation works with a subset of three of these relationships; proximity, similarity, and continuation. Incorporating this perceptual organizing information into a structured, symbolic scene representation strongly increases the available information about the scene. This information can be used to improve pattern recognition, and extensions may address problems in 3-D modeling, multi-temporal correlations, and multisensor information fusion.