Building information modeling (BIM), with detailed geometry and semantics of the indoor environment, has become an essential part of smart city development and city information modeling (CIM). However, visualizing large-scale BIM models within geographic information systems (GIS), such as virtual globes, remains a technological challenge with limited hardware resources. Previous methods generally removed indoor features in a single-source (BIM) scene to reduce the computational burden from outdoor views, which have not been applied to the multi-source and -scale geographic environment (e.g., virtual globes). This approach neglected special BIM semantics (e.g., transparent windows), which may miss a part of geographic features or buildings and cause unreasonable visualization. Besides, the method overlooked indoor visualization optimization, which may burden computing resources when visualizing big and complex buildings from indoor views. To address these problems, we propose a semantics-based method for visualizing large-scale BIM models within indoor and outdoor environments. First, we organize large-scale BIM models based on a latitude-longitude grid (LLG) in the outdoor environment; a multilayer cell-and-portal graph is used to index the structure of the BIM model and building entities. Second, we propose a scheduling algorithm to achieve the integrated visualization in indoor and outdoor environments considering BIM semantics. The application of the proposed method to a multi-scale and -source environment confirmed that it can achieve an effective and efficient visualization for huge BIM models in indoor-outdoor scenes. Compared with the previous study, the proposed method considers the BIM semantics and thus can visualize more complete features from outdoor and indoor views of BIM models in the virtual globe. Besides, the study only loads as visible data as possible, which can retain lower the volume of increased geometry, and thus keep a higher frame rate for the tested areas.