Adaptive ε-Greedy Exploration in Reinforcement Learning Based on Value Differences

Tokic, Michel

doi:10.1007/978-3-642-16111-7_23

Cited by 211 publications

(139 citation statements)

References 8 publications

Supporting

Mentioning

136

Contrasting

Unclassified

Order By: Relevance

“…Most often, -greedy method is selected since it is not required to memorize any exploration specific data and also in many applications it achieved near optimal results [40].…”

Section: Exploration-exploitation Strategymentioning

confidence: 99%

Adaptive Image Thresholding of Yellow Peppers for a Harvesting Robot

2018

View full text Add to dashboard Cite

Abstract:The presented work is part of the H2020 project SWEEPER with the overall goal to develop a sweet pepper harvesting robot for use in greenhouses. As part of the solution, visual servoing is used to direct the manipulator towards the fruit. This requires accurate and stable fruit detection based on video images. To segment an image into background and foreground, thresholding techniques are commonly used. The varying illumination conditions in the unstructured greenhouse environment often cause shadows and overexposure. Furthermore, the color of the fruits to be harvested varies over the season. All this makes it sub-optimal to use fixed pre-selected thresholds. In this paper we suggest an adaptive image-dependent thresholding method. A variant of reinforcement learning (RL) is used with a reward function that computes the similarity between the segmented image and the labeled image to give feedback for action selection. The RL-based approach requires less computational resources than exhaustive search, which is used as a benchmark, and results in higher performance compared to a Lipschitzian based optimization approach. The proposed method also requires fewer labeled images compared to other methods. Several exploration-exploitation strategies are compared, and the results indicate that the Decaying Epsilon-Greedy algorithm gives highest performance for this task. The highest performance with the Epsilon-Greedy algorithm ( = 0.7) reached 87% of the performance achieved by exhaustive search, with 50% fewer iterations than the benchmark. The performance increased to 91.5% using Decaying Epsilon-Greedy algorithm, with 73% less number of iterations than the benchmark.

show abstract

“…Most often, -greedy method is selected since it is not required to memorize any exploration specific data and also in many applications it achieved near optimal results [40].…”

Section: Exploration-exploitation Strategymentioning

confidence: 99%

Adaptive Image Thresholding of Yellow Peppers for a Harvesting Robot

2018

View full text Add to dashboard Cite

show abstract

“…Exploration strategies conventionally use some form of the posterior variance [7] or the information-entropy [8] to quantify the information gained by a given action. While both strategies make intuitive sense, they are mathematically imprecise descriptors of the information gain which, due to a theorem of uniqueness in information theory [1], is defined as…”

Section: A Information Gain and Explorationmentioning

confidence: 99%

Data-driven prediction of EVAR with confidence in time-varying datasets

Axelrod

Carlone

Chowdhary

et al. 2016

2016 IEEE 55th Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

Abstract-The key challenge for learning-based autonomous systems operating in time-varying environments is to predict when the learned model may lose relevance. If the learned model loses relevance, then the autonomous system is at risk of making wrong decisions. The entropic value at risk (EVAR) is a computationally efficient and coherent risk measure that can be utilized to quantify this risk. In this paper, we present a Bayesian model and learning algorithms to predict the state-dependent EVAR of time-varying datasets. We discuss applications of EVAR to an exploration problem in which an autonomous agent has to choose a set of sensing locations in order to maximize the informativeness of the acquired data and learn a model of an underlying phenomenon of interest. We empirically demonstrate the efficacy of the presented model and learning algorithms on four real-world datasets.

show abstract

“…We chose to model our situation as a multi-armed bandit problem [22]. There is a room full of casino machines (the bandits) with different reward distributions.…”

Section: Reinforcement Learningmentioning

confidence: 99%

Scalable, generic, and adaptive systems for focused crawling

Gouriten

Maniu

Senellart

2014

Proceedings of the 25th ACM Conference on Hypertext and Social Media

View full text Add to dashboard Cite

Focused crawling is the process of exploring a graph iteratively, focusing on parts of the graph relevant to a given topic. It occurs in many situations such as a company collecting data on competition, a journalist surfing the Web to investigate a political scandal, or an archivist recording the activity of influential Twitter users during a presidential election. In all these applications, users explore a graph (e.g., the Web or a social network), nodes are discovered one by one, the total number of exploration steps is constrained, some nodes are more valuable than others, and the objective is to maximize the total value of the crawled subgraph.In this article, we introduce scalable, generic, and adaptive systems for focused crawling. Our first effort is to define an abstraction of focused crawling applicable to a large domain of real-world scenarios. We then propose a generic algorithm, which allows us to identify and optimize the relevant subsystems. We prove the intractability of finding an optimal exploration, even when all the information is available.Taking this intractability into account, we investigate how the crawler can be steered in several experimental graphs. We show the good performance of a greedy strategy and the importance of being able to run at each step a new estimation of the crawling frontier. We then discuss this estimation through heuristics, selftrained regression, and multi-armed bandits. Finally, we investigate their scalability and efficiency in different real-world scenarios and by comparing with state-of-the-art systems.

show abstract

Adaptive ε-Greedy Exploration in Reinforcement Learning Based on Value Differences

Cited by 211 publications

References 8 publications

Adaptive Image Thresholding of Yellow Peppers for a Harvesting Robot

Adaptive Image Thresholding of Yellow Peppers for a Harvesting Robot

Data-driven prediction of EVAR with confidence in time-varying datasets

Scalable, generic, and adaptive systems for focused crawling

Contact Info

Product

Resources

About