We report our experience in implementing UbiCrawler, a scalable distributed Web crawler, using the Java programming language. The main features of UbiCrawler are platform independence, linear scalability, graceful degradation in the presence of faults, a very effective assignment function (based on consistent hashing) for partitioning the domain to crawl, and more in general the complete decentralization of every task. The necessity of handling very large sets of data has highlighted some limitations of the Java APIs, which prompted the authors to partially reimplement them. Copyright © 2004 John Wiley & Sons, Ltd.

We consider the problem of computing market equilibria and show three results. (i) For exchange economies satisfying weak gross substitutability we analyze a simple discrete version of tâtonnement, and prove that it converges to an approximate equilibrium in polynomial time. This is the first polynomial-time approximation scheme based on a simple tâtonnement process. It was only recently shown, using vastly more sophisticated techniques, that an approximate equilibrium for this class of economies is computable in polynomial time. (ii) For Fisher's model, we extend the frontier of tractability by developing a polynomial-time algorithm that applies well beyond the homothetic case and the gross substitutes case. (iii) For production economies, we obtain the first polynomial-time algorithms for computing an approximate equilibrium when the consumers' side of the economy satisfies weak gross substitutability and the producers' side is restricted to positive production.

We consider Leontief exchange economies, i.e., economies where the consumers desire goods in fixed proportions. Unlike bimatrix games, such economies are not guaranteed to have equilibria in general. On the other hand, they include suitable restricted versions which always have equilibria.We give a reduction from two-player games to a special family of Leontief exchange economies, which are guaranteed to have equilibria, with the property that the Nash equilibria of any game are in one-to-one correspondence with the equilibria of the corresponding economy.Our reduction exposes a potential hurdle inherent in solving certain families of market equilibrium problems: finding an equilibrium for Leontief economies (where an equilibrium is guaranteed to exist) is at least as hard as finding a Nash equilibrium for two-player nonzero sum games.As a corollary of the one-to-one correspondence, we obtain a number of hardness results for questions related to the computation of market equilibria, using results already established for games [17]. In particular, among other results, we show that it is NP-hard to say whether a particular family of Leontief exchange economies, that is guaranteed to have at least one equilibrium, has more than one equilibrium.Perhaps more importantly, we also prove that it is NPhard to decide whether a Leontief exchange economy has an equilibrium. This fact should be contrasted against the known PPAD-completeness result of [30], which holds when the problem satisfies some standard sufficient conditions that make it equivalent to the computational version of Brouwer's Fixed Point Theorem.On the algorithmic side, we present an algorithm for finding an approximate equilibrium for some special Leontief economies, which achieves quasi-polynomial time whenever each trader does not demand too much more of any good than some other good.

No abstract