By interpreting spatial relations among objects, many applications such as video surveillance, robotics, and scene understanding systems can be utilized efficiently for different purposes. The vast majority of known models for spatial relationships are carried out with an image. However, due to the advance in technology, a three-dimensional scene became available. For our knowledge, most of the interpreted spatial relations were defined between silent objects in images. A technique for determining the dynamic spatial relation between a moving object and another silent one in a time varying scene is presented here. The spatial relationships were determined by using motion-based object tracking along with hypergraph object-oriented model. Defining the spatial relationship types between a single silent object and a moving human body has applied based on two strategies; determining each object with a bounding box, then comparing the locations of these boxes by applying certain conditional rules. This study identifies some of the spatial relationships in three dimensions of streaming frames, which has carried out by establishing a highly accurate and efficient proposed algorithm. The following relations have been studied; (“direct in front of”, “in front of on the Right/Left”, “direct behind of”, “behind of on the Right/Left”, “to the Right”, “to the Left”, “On”, “Under”, Besides, and “Besides on to the Right/Left”). The experimental results, which have been obtained based on actual indoor streaming frames, show effectiveness and reliable execution of our system