In this talk we present a combination of well-known and proven HCI techniques to create a markerless performance capturing system based on low-cost consumer hardware for live performances with virtual characters. The use case for our approach was a theatrical play on the occasion of the celebration of the 60 th anniversary 1 of the federal state of Baden-Wuerttemberg, Germany, in which a virtual alter ego of the current prime minister interacted directly with the stage actors and musicians.