In this paper, we address the tradeoff between exploration and exploitation for agents which need to learn more about the structure of their environment in order to perform more effectively. For example, a software agent operating on the World Wide Web may need to learn which sites on the net are most useful, and the most efficient routes to those sites. We compare exploration strategies for a repeated task, where the agent is given some particular task to perform some number of times. Tasks are modeled as navigation on a partially known (deterministic) graph. This paper describes a new utilitybased exploration algorithm for repeated tasks which interleaves exploration with task performance. The method takes into account both the costs and the potential benefits (for future task repetitions) of different exploratory actions. Exploration is performed in a greedy fashion, with the locally optimal exploratory action performed during repetition of each task. We experimentally evaluated our utility-based interleaved exploration algorithm against a heuristic search algorithm for exploration before task performance (a priori exploration) as well as a randomized interleaved exploration algorithm. We found that for a single repeated task, utility-based interleaved exploration consistently outperforms the alternatives, unless the number of task repetitions is very high. In addition, we extended the algorithms for the case of multiple repeated tasks, where the agent has a different, randomly-chosen task (from a known subset of possible tasks) to perform each time. Here too, we found that utility-based interleaved exploration is clear in most cases. Int. J. Patt. Recogn. Artif. Intell. 1999.13:963-986. Downloaded from www.worldscientific.com by FLINDERS UNIVERSITY LIBRARY on 02/03/15. For personal use only.

No abstract

Multiphase flow of oil, gas, and water occurs in a reservoir’s underground formation and also within the associated downstream pipeline and structures. Computer simulations of such phenomena are essential in order to achieve the behavior of parameters including but not limited to evolution of phase fractions, temperature, velocity, pressure, and flow regimes. However, within the oil and gas industry, due to the highly complex nature of such phenomena seen in unconventional assets, an accurate and fast calculation of the aforementioned parameters has not been successful using numerical simulation techniques, i.e., computational fluid dynamic (CFD). In this study, a fast-track data-driven method based on artificial intelligence (AI) is designed, applied, and investigated in one of the most well-known multiphase flow problems. This problem is a two-dimensional dam-break that consists of a rectangular tank with the fluid column at the left side of the tank behind the gate. Initially, the gate is opened, which leads to the collapse of the column of fluid and generates a complex flow structure, including water and captured bubbles. The necessary data were obtained from the experience and partially used in our fast-track data-driven model. We built our models using Levenberg Marquardt algorithm in a feed-forward back propagation technique. We combined our model with stochastic optimization in a way that it decreased the absolute error accumulated in following time-steps compared to numerical computation. First, we observed that our models predicted the dynamic behavior of multiphase flow at each time-step with higher speed, and hence lowered the run time when compared to the CFD numerical simulation. To be exact, the computations of our models were more than one hundred times faster than the CFD model, an order of 8 h to minutes using our models. Second, the accuracy of our predictions was within the limit of 10% in cascading condition compared to the numerical simulation. This was acceptable considering its application in underground formations with highly complex fluid flow phenomena. Our models help all engineering aspects of the oil and gas industry from drilling and well design to the future prediction of an efficient production.