HDSKG: Harvesting domain specific knowledge graph from content of webpages

Abstract

Knowledge graph is useful for many different domains like search result ranking, recommendation, exploratory search, etc. It integrates structural information of concepts across multiple information sources, and links these concepts together. The extraction of domain specific relation triples (subject, verb phrase, object) is one of the important techniques for domain specific knowledge graph construction. In this research, an automatic method named HDSKG is proposed to discover domain specific concepts and their relation triples from the content of webpages. We incorporate the dependency parser with rule-based method to chunk the relations triple candidates, then we extract advanced features of these candidate relation triples to estimate the domain relevance by a machine learning algorithm. For the evaluation of our method, we apply HDSKG to Stack Overflow (a Q&A website about computer programming). As a result, we construct a knowledge graph of software engineering domain with 35279 relation triples, 44800 concepts, and 9660 unique verb phrases. The experimental results show that both the precision and recall of HDSKG (0.78 and 0.7 respectively) is much higher than the openIE (0.11 and 0.6 respectively). The performance is particularly efficient in the case of complex sentences. Further more, with the self-training technique we used in the classifier, HDSKG can be applied to other domain easily with less training data.

From Vision to Content: Construction of Domain-Specific Multi-Modal Knowledge Graph

“…Among them, [40]- [42] is to extract structured knowledge from domain-related texts to construct domain knowledge graph. However, HDSKG [44] extracts relational triples from Web pages and then uses a pre-trained SVM classifier and domain dictionary to determine the domain relevance of the extracted triples. MeSH-like [45] extracts entity attributes from medical textbooks, medical online websites, and other domain-related texts using a rule-based approach and fuses them with SimHash-TF-ID algorithm.…”

Constructing a Knowledge Base of Coding Conventions from Online Resources

“…The syntactic analysis approach [15] is widely used to extract <subject, predicate, object> triples from sentences, but it isn't applicable for constructing CCBase. It is because entities and property values of coding conventions could not be directly collected from sentences, and also predicates in triples parsed by syntactic analysis approach could not be used as relations in CCBase.…”

“…We construct CCBase in a top-down way, extracting entities and relations from unstructured documents guided by ontology. To evaluate the effectiveness of this method, we compare it with two bottom-up extraction methods: the popular open information extraction toolopen IE [20] and a domainspecific extraction method -HDSKG [15]. Three popular metrics are selected: precision, recall and F1 score.…”

Developing a Quick Response Product Configuration System under Industry 4.0 Based on Customer Requirement Modelling and Optimization Method

“…These high-complexity/low-volume environments driven by Industry 4.0 trend and techniques represent a very difficult and challenging situation for production companies [12][13][14][15]. In today's practice, small and medium-sized enterprises (SME) can apply smart factory production line solution embedded with IoT and CPS technology to embrace the opportunity of small lot sizes, different product lines with low budgets for automation investments [8,12,[16][17][18]. By equipping this, the gap experienced in today's manufacturing environment can be filled and a kind of highly flexible, productive, high-complexity/low-volume, and SME-friendly can be a new paradigm [8,13].…”