A real-time retrieval system using change aware distributed file system

Abstract

Intranet information retrieval is very important to discover the useful knowledge. In this process, search engine is useful. However, conventional search engines, which are based on centralized architecture, are not suited for the intranet information retrieval because intranet information is frequently updated. Centralized search engines take a long time to collect web pages by robots. So, we have developed a distributed search engine, Cooperative Search Engine (CSE), to retrieve fresh information. In CSE, a local search engine located in each Web server makes an index of local pages. And, a Meta search server integrates these local search engines to realize a global search engine. CSE takes a few minutes to update index because each local search engine scans all files in its own disk. In this paper, we propose the notification mechanism using change aware distributed file system. As this result, we realize the real-time information retrieval.