We present a smartphone-based indoor localisation system, able to track pedestrians over multiple floors. The system uses Pedestrian Dead Reckoning (PDR), which exploits data from the smartphone’s inertial measurement unit to estimate the trajectory. The PDR output is matched to a scaled floor plan and fused with model-based WiFi received signal strength fingerprinting by a Backtracking Particle Filter (BPF). We proposed a new Viterbi-based floor detection algorithm, which fuses data from the smartphone’s accelerometer, barometer and WiFi RSS measurements to detect stairs and elevator usage and to estimate the correct floor number. We also proposed a clustering algorithm on top of the BPF to solve multimodality, a known problem with particle filters. The proposed system relies on only a few pre-existing access points, whereas most systems assume or require the presence of a dedicated localisation infrastructure. In most public buildings and offices, access points are often available at smaller densities than used for localisation. Our system was extensively tested in a real office environment with seven 41 m × 27 m floors, each of which had two WiFi access points. Our system was evaluated in real-time and batch mode, since the system was able to correct past states. The clustering algorithm reduced the median position error by 17% in real-time and 13% in batch mode, while the floor detection algorithm achieved a 99.1% and 99.7% floor number accuracy in real-time and batch mode, respectively.