Agent-based architectures and Intelligent Virtual Agents have been used to support education and exploration of objects and places of interest such as exhibitions, museums and historical or cultural sites. We present a number of issues from this body of work including: agent versus user control, education versus entertainment, navigation of semantically enriched objects, believable agents and the role of explanation. Key findings are applied to the design of user-agent interactions for Virtual Saarlouis, an historic garrison town.