The simulation of synthetic humans inhabiting virtual environments is a current research topic with a great number of behavioral problems to be tackled. Semantical virtual environments (SVEs) have recently been proposed not only to ease world modeling but also to enhance the agent–object and agent–agent interaction. Thus, we propose the use of ontologies to define the world’s knowledge base and to introduce semantic levels of detail that help the sensorization of complex scenes—containing lots of interactive objects. The object taxonomy also helps to create general and reusable operativity for autonomous characters—for example, liquids can be poured from containers such as bottles. On the other hand, we use the ontology to define social relations among agents within an artificial society. These relations must be taken into account in order to display socially acceptable decisions. Therefore, we have implemented a market-based social model that reaches coordination and sociability by means of task exchanges. This paper presents a multi-agent framework oriented to simulate socially intelligent characters in SVEs. The framework has been successfully tested in three-dimensional (3D) dynamic scenarios while simulating a virtual university bar, where groups of waiters and customers interact with both the objects in the scene and the other virtual agents, finally displaying complex social behaviors.