The design intent and many meaningful semantics in Building Information Modeling (BIM) models are often implicit, and some explicit semantics are lost during model exchanges. These missing information can be artificially supplemented through a process called semantic enrichment. However, previous research on semantic enrichment has primarily focused on specific tasks, leading to a limited scope of predictions, and lacks comprehensive, seamless approaches. In this study, we aim to infer BIM semantics from fundamental model data, pure object geometries, and organize the predicted results into a graph-based Common Data Environment (CDE) to support intelligent applications. Consequently, we propose a framework of generic semantic enrichment which includes four fundamental tasks in the context of graphs and a process control mechanism to execute a set of tools in a proper sequence. To validate its feasibility, we selected a real-world apartment model and developed six tools to generate the graph-based CDE from its object geometries. Additionally, applications were implemented on the graph-based CDE, such as 1) enriching 1Wang,