Recently, so-called smart speakers have been introduced and they include a microphone array. One potential application of such a smart speaker is to use it for calibrating a larger audio system which the speaker is a part of. In this paper, we propose a method to perform this calibration using one or several smart speakers. Specifically, a map is estimated of the sensors and sound sources. As opposed to existing methods, the proposed method can create this map for both synchronised and unsynchronised sound sources by taking the different localisation errors into account. We show that this gives more accurate estimates than assuming identical estimation errors, and that existing methods are outperformed in terms of estimation accuracy for various noise levels and reverberation times.