Semantic annotation provides a fundamental technology for web-based visualization application. However, the lack of automatic reasoning ability makes annotation task both boring and time-consuming when mass 3D models occurs. By means of three-layer ontology based on X3D, a comprehensive approach is proposed to implement semi-automatic semantic annotation in web3D scene. Firstly, three-layer ontology including general ontology, domain ontology and scene ontology is constructed by combining X3D instances with general knowledge, which forms a comprehensive knowledge representation for semantic annotation. Then, based on X3D, features extraction, node reduction and node matching is carried out to implement part-annotation of a certain scene. Next, based on the rules defined, annotation spread mechanism is carried out to implement automatic annotation. Lastly, a case study based on implemented prototype system is also provided to demonstrate the process of semantic annotation. The result shows that the proposed approach can help to implement semantic annotation effectively.