An Overview of Schema Extraction and Matching Techniques

Hassan, Muhammad Umair; Shankat, Kamran; Niu, Dongmie; Mahreen, Sundas; Ma, Yingjun; Haider, Fatima; Mubashir, Muhammad; Zhao, Xiuyang

doi:10.1109/imcec.2018.8469502

Cited by 3 publications

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A large amount of relational data exist as tables which are often hyperlinked, providing additional meaning to the data. Lots of work has been done to improve structured extraction [25,37,38] and data cleaning [27]; however, there is still room for improvement in extraction tools. The biggest and most fundamental challenge which has not been fully achieved yet is maintaining the coverage and accuracy of the extraction of a schema for hyperlinked tables.…”

Section: Discussionmentioning

confidence: 99%

A Novel Approach to Data Extraction on Hyperlinked Webpages

2019

View full text Add to dashboard Cite

The World Wide Web has an enormous amount of useful data presented as HTML tables. These tables are often linked to other web pages, providing further detailed information to certain attribute values. Extracting schema of such relational tables is a challenge due to the non-existence of a standard format and a lack of published algorithms. We downloaded 15,000 web pages using our in-house developed web-crawler, from various web sites. Tables from the HTML code were extracted and table rows were labeled with appropriate class labels. Conditional random fields (CRF) were used for the classification of table rows, and a nondeterministic finite automaton (NFA) algorithm was designed to identify simple, complex, hyperlinked, or non-linked tables. A simple schema for non-linked tables was extracted and for the linked-tables, relational schema in the form of primary and foreign-keys (PK and FK) were developed. Child tables were concatenated with the parent table’s attribute value (PK), serving as foreign keys (FKs). Resultantly, these tables could assist with performing better and stronger queries using the join operation. A manual checking of the linked web table results revealed a 99% precision and 68% recall values. Our 15,000-strong downloadable corpus and a novel algorithm will provide the basis for further research in this field.

show abstract

Section: Discussionmentioning

confidence: 99%

A Novel Approach to Data Extraction on Hyperlinked Webpages

2019

View full text Add to dashboard Cite

show abstract

Restoration of Data Structures Using Machine Learning Techniques

Cvijetić,

Radivojević

2023

IEEE Access

View full text Add to dashboard Cite

Tabular data is the most common format used to represent real-world information. Almost all programs created for storing or processing data, such as relational database systems, spreadsheets, and statistical analysis software can import or export tabular data. These programs are not sufficiently robust to automatically solve the problems of importing messy delimited files or files that contain data from multiple tables. Additional messy datasets contain data delimited by multiple delimiters without the names of the table columns, and parts of the table rows have substituted or deleted columns. This paper proposes the STCExtract algorithm for reconstructing table structures and data in which the input file can be arranged. The STCExtract algorithm is designed to be domain-independent and modular according to machine learning algorithms and other parameters. The algorithm was developed as a two-phase process, in which the original data tables were recognized in the first phase and the columns of the original data tables in the second phase. The STCExtract algorithm was evaluated through expensive experiments using multiple real datasets. Multiple messy datasets were generated for the four experiments. Three experiments were conducted to determine the optimal parameters for the STCExtract algorithm. A fourth experiment was conducted to evaluate the proposed algorithm. The results show that the STCExtract algorithm correctly arranged the structure of the tables with an accuracy of 94.4% to 100%. The accuracy of the STCExtract algorithm in the second phase (when the data were allocated to columns) ranged from 59.7% to 90.2%.

show abstract