The web-DL environment for building digital libraries from the web

Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Moura

et al. 2006

Many Web IR and Digital Library applications require a crawling process to collect pages with the ultimate goal of taking advantage of useful information available on Web sites. For some of these applications the criteria to determine when a page is to be present in a collection are related to the page content. However, there are situations in which the inner structure of the pages provides a better criteria to guide the crawling process than their content. In this paper, we present a structure-driven approach for generating Web crawlers that requires a minimum effort from users. The idea is to take as input a sample page and an entry point to a Web site and generate a structure-driven crawler based on navigation patterns, sequences of patterns for the links a crawler has to follow to reach the pages structurally similar to the sample page. In the experiments we have carried out, structure-driven crawlers generated by our new approach were able to collect all pages that match the samples given, including those pages added after their generation.

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Structure-driven crawler generation by example

Vidal

Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

Moura

et al. 2006

“…The usefulness and feasibility of some of our data extraction tools have been demonstrated with the development of Web-DL, an environment for constructing digital libraries from bibliographic data available on the Web [29]. It operates by allowing one to collect and extract bibliographic data from the Web, standardize it, and publish it through a digital library system.…”

Section: B Web Data Extractionmentioning

confidence: 99%

Cooperative Research on Web Data Management at UFMG and UFAM - A Brief Report

Laender

2008 Latin American Web Conference

2008

The World Wide Web has become a huge repository of data of interest for a variety of application domains. However, the same features that have made the Web so useful and popular also impose important restrictions on the way the data it contains can be manipulated. Particularly, in the traditional Web scenario, there is an inherent difficulty in gaining access to data that is implicitly present in Web pages but is not readily available. The term Web Data Management (WDM) has been used to refer to the study of problems related to fetching, extracting, querying, modeling, storing, transforming, and integrating data available on the Web. These issues have been growing in importance in the scientific community in the last years, as it can be be seen by the considerable space devoted to them in important publication venues. This interest is justified not only by the scientific and technical challenges involved in WDM problems, but also, and specially, by the growing demand from the industry for solving such problems. In this paper, we present a brief report on the WDM cooperative research carried out by the Database Laboratory at the Federal University of Minas Gerais (LBD/UFMG) and the Information Technology Group at the Federal University of Amazonas (GTI/UFAM). The main aim of this research effort is to develop methods and tools for dealing with data available on the Web and in other non-structured sources (e.g., XML documents), thus providing facilities similar to those available in traditional database systems for managing such data.

“…Calado et al [3] desenvolveram um ambiente denominado Web-DL, que permite criar uma biblioteca digital a partir de documentos publicados na web. O sistema foi testado, com sucesso, na Networked Digital Library of Theses and Dissertations (NDLTD) [11].…”

Section: Trabalhos Relacionadosunclassified

Freebie

Proceedings of the 12th Brazilian Symposium on Multimedia and the Web

Baptista

Schiel

et al. 2006

Nowadays there is an increasing volume of heterogeneous documents in a company, in different media such as texts, spreadsheets, images, maps, etc. Most of these documents contain relevant information about the company; hence they need to be stored, indexed and retrieved efficiently. Digital libraries aim to address this issue by providing an infrastructure which enables to manipulate these documents. Furthermore, the use of geographical location of such documents has become more attractive recently, especially due to the dissemination of GPS units in many devices. Thus, these documents may have spatial footprints which allow for the enhancement of the quality of search in such digital libraries. This paper presents Freebie, an open source digital library, which provides both textual and spatial indexing on underlying documents. The spatial metadata may be used regarding the subject and author location; and the place of the publication.