Indoor environment inference is of great importance to mobile and pervasive computing. As high-level metadata of indoor environment, floor maps contain rich information and are widely required in many pervasive systems. However, despite significant research progress, automatic inference of indoor maps has been less studied.In this paper, we present iMap, a smartphone-based opportunistic sensing system that automatically constructs the indoor maps by merging crowdsourced walking trajectories from smartphone users. Most importantly, indoor semantics, such as stairs, escalators, elevators and doors are also automatically detected and annotated to the constructed map in the same inference process. The evaluation result shows that iMap can accurately detect different indoor semantics and be applied to different indoor environments. With the capability of generating semanticannotated indoor maps without requiring any prior knowledge of the indoor environment, iMap has the potential to be widely deployed in practice.