Building an open source meta-search engine

Gullì, Antonio; Signorini, Alessio

doi:10.1145/1062745.1062840

Cited by 27 publications

(20 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The term metacrawling on the classical Web is usually discussed under the term "meta search engine" [15]- [17]. Retrieving and combining results from search engines in the classical Web domain is not something new, and meta search engines include SavvySearch [18], Helios [19], and WebCrawler. However, on the Semantic Web domain, meta search is not currently employed on any of the existing semantic search engines [20]- [24].…”

Section: Semantic Web Crawling and Metacrawlingmentioning

confidence: 99%

SpEnD: Linked Data SPARQL Endpoints Discovery Using Search Engines

Yumuşak

Doğdu

Kodaz

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYLinked data endpoints are online query gateways to semantically annotated linked data sources. In order to query these data sources, SPARQL query language is used as a standard. Although a linked data endpoint (i.e. SPARQL endpoint) is a basic Web service, it provides a platform for federated online querying and data linking methods. For linked data consumers, SPARQL endpoint availability and discovery are crucial for live querying and semantic information retrieval. Current studies show that availability of linked datasets is very low, while the locations of linked data endpoints change frequently. There are linked data respsitories that collect and list the available linked data endpoints or resources. It is observed that around half of the endpoints listed in existing repositories are not accessible (temporarily or permanently offline). These endpoint URLs are shared through repository websites, such as Datahub.io, however, they are weakly maintained and revised only by their publishers. In this study, a novel metacrawling method is proposed for discovering and monitoring linked data sources on the Web. We implemented the method in a prototype system, named SPARQL Endpoints Discovery (SpEnD). SpEnD starts with a "search keyword" discovery process for finding relevant keywords for the linked data domain and specifically SPARQL endpoints. Then, the collected search keywords are utilized to find linked data sources via popular search engines (Google, Bing, Yahoo, Yandex). By using this method, most of the currently listed SPARQL endpoints in existing endpoint repositories, as well as a significant number of new SPARQL endpoints, have been discovered. We analyze our findings in comparison to Datahub collection in detail.

show abstract

Section: Semantic Web Crawling and Metacrawlingmentioning

confidence: 99%

SpEnD: Linked Data SPARQL Endpoints Discovery Using Search Engines

Yumuşak

Doğdu

Kodaz

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…Aggregated search can be compared to federated search [12], metasearch [7] and distributed information retrieval [5], which are all concerned with the retrieval of documents from several sources (e.g. digital libraries, search engines, specialist collections, etc.).…”

Section: Related Workmentioning

confidence: 99%

Factors affecting click-through behavior in aggregated search interfaces

Sushmita

Joho

Lalmas

et al. 2010

Proceedings of the 19th ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

An aggregated search interface is designed to integrate search results from different sources (web, image, video, blog, etc) into a single result page. This paper presents two user studies investigating factors affecting users click-through behavior on aggregated search interfaces. We tested two aggregated search interfaces: one where results from the different sources are blended into a single list (called blended ), and another, where results from each source are presented in a separate panel (called non-blended ). A total of 1,296 search sessions performed by 48 participants were analysed in our study. Our results suggest that 1) the position of search results is significant only in the blended and not in the nonblended design; 2) participants' click-through behavior on videos is different from other sources; and finally 3) capturing a task's orientation towards particular sources is an important factor for further investigation and research.

show abstract

“…It returns final web documents ranked by relevance. In the Helios architecture [1] the MSE system uses standard merger and ranker modules. To achieve high performance it utilizes async I/O and parallel TCP connections with the remote search engines.…”

Section: Introductionmentioning

confidence: 99%

“…same with i r in (1). Unlike the KE algorithm, the documents are here ranked in descending order (the higher the weight the better the ranking score).…”

Section: Introductionmentioning

confidence: 99%

The Analysis of Rank Fusion Techniques to Improve Query Relevance

et al. 2015

View full text Add to dashboard Cite

Rank fusion meta-search engine algorithms can be used to merge web search results of multiple search engines. In this paper we introduce two variants of the IntroductionThere are many proposals for a meta-search engine (MSE). Given a query (a set of keywords), typically an MSE system retrieves web pages that are relevant to the query by exploiting all its underlying search engines. It sends the query to these engines; the results obtained are then merged and ranked. It returns final web documents ranked by relevance. In the Helios architecture [1] the MSE system uses standard merger and ranker modules. To achieve high performance it utilizes async I/O and parallel TCP connections with the remote search engines. In the Tadpole architecture [2,3], the rank fusion algorithms are based on a variety of parameters, such as the rank order and the number of times an URL appears in the results of each of its search engines components, to compute a weight for each collected results [2][3][4]. There is also the concern of user specific needs. For example, an MSE should ideally let the user choose his favourite search engines from an available list, and do query modifications, as well as explore available rank fusion techniques [2].In general, rank fusion algorithms offer improvement of the relevance scores of the returned documents of multiple search engines. Dwork, Kumar, Naor, and Sivakumar propose the use of rank aggregation methods for MSEs viz. the Borda's method, Footrule and Scaled Footrule, and Markov Chain methods [5]. Lam and Leung propose a complete directed graph viz. MST Algorithm [6]. Supervised rank aggregation methods such as Borda-Fuse and supervised Markov Chain based methods are investigated in [7]. KE algorithm [8] and its variants [4] exploit the ranking on the results that an MSE receives from its component engines, by considering the number of documents appearances in the component engines' lists with equal reliability assumption of those engines. Another rank fusion MSE algorithm named Count Function [9] defines web documents ranking as summing ranks as per positions of a URL

show abstract

Building an open source meta-search engine

Cited by 27 publications

References 7 publications

SpEnD: Linked Data SPARQL Endpoints Discovery Using Search Engines

SpEnD: Linked Data SPARQL Endpoints Discovery Using Search Engines

Factors affecting click-through behavior in aggregated search interfaces

The Analysis of Rank Fusion Techniques to Improve Query Relevance

Contact Info

Product

Resources

About