Projective simulation applied to the grid-world and the mountain-car problem

Melnikov, A. A.; Makmal, Adi; Briegel, Hans J.

doi:10.5430/air.v3n3p24

Cited by 17 publications

(35 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to this fact, this graph does not have a tree structure but rather resembles a maze. Navigating in a maze, in turn, constitutes one of the classic textbook RL problems [10,42,50,51]. Second, our empirical analysis suggests that experiments generating high-dimensional multipartite entanglement tend to have some structural similarities [12] (see Fig.…”

Section: Ingredients For Successful Learningmentioning

confidence: 83%

“…PS is a physics-motivated framework which can be used to construct RL agents. PS was shown to perform well in standard RL problems [41][42][43][44], in advanced robotics applications [45], and it is also amenable for quantum enhancements [46][47][48]. The main component of the PS agent is its memory network (shown in Fig.…”

Section: Resultsmentioning

confidence: 99%

“…Second, our empirical analysis suggests that experiments generating high-dimensional multipartite entanglement tend to have some structural similarities [12] (see Fig. 3(a)- The depicted colored maze represents an analogy between the task of finding the shortest implementation of an experiment and the task of navigating in a maze [10,42,50,51]. Arrows of different colors represent distinct optical elements that are placed in the experiment.…”

Section: Ingredients For Successful Learningmentioning

confidence: 97%

See 2 more Smart Citations

Active learning machine learns to create new quantum experiments

Melnikov

Nautrup

Krenn

et al. 2018

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

299

262

View full text Add to dashboard Cite

How useful can machine learning be in a quantum laboratory? Here we raise the question of the potential of intelligent machines in the context of scientific research. A major motivation for the present work is the unknown reachability of various entanglement classes in quantum experiments. We investigate this question by using the projective simulation model, a physics-oriented approach to artificial intelligence. In our approach, the projective simulation system is challenged to design complex photonic quantum experiments that produce high-dimensional entangled multiphoton states, which are of high interest in modern quantum experiments. The artificial intelligence system learns to create a variety of entangled states, and improves the efficiency of their realization. In the process, the system autonomously (re)discovers experimental techniques which are only now becoming standard in modern quantum optical experiments -a trait which was not explicitly demanded from the system but emerged through the process of learning. Such features highlight the possibility that machines could have a significantly more creative role in future research.

show abstract

Section: Ingredients For Successful Learningmentioning

confidence: 83%

Section: Resultsmentioning

confidence: 99%

Section: Ingredients For Successful Learningmentioning

confidence: 97%

See 1 more Smart Citation

Active learning machine learns to create new quantum experiments

Melnikov

Nautrup

Krenn

et al. 2018

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

299

262

View full text Add to dashboard Cite

show abstract

“…A special case of the RPS agents that we have considered in section 4 is obtained by considering the reflective analog of so-called 'two-layered' PS agents, where all transition are one-step transitions from percepts to actions [11]. Such agents have a very simple structure, yet were shown to be capable of learning to solve non-trivial environmental tasks [15,25]. In the RPS analog of two-layered PS agents [11], the associated Markov chains of each percept-specific clip network are rank-one throughout the entire learning process of the agent.…”

Section: A1 Rank-one Reflecting Psmentioning

confidence: 99%

Quantum-enhanced deliberation of learning agents using trapped ions

2015

View full text Add to dashboard Cite

A scheme that successfully employs quantum mechanics in the design of autonomous learning agents has recently been reported in the context of the projective simulation (PS) model for artificial intelligence. In that approach, the key feature of a PS agent, a specific type of memory which is explored via random walks, was shown to be amenable to quantization, allowing for a speed-up. In this work we propose an implementation of such classical and quantum agents in systems of trapped ions. We employ a generic construction by which the classical agents are 'upgraded' to their quantum counterparts by a nested process of adding coherent control, and we outline how this construction can be realized in ion traps. Our results provide a flexible modular architecture for the design of PS agents. Furthermore, we present numerical simulations of simple PS agents which analyze the robustness of our proposal under certain noise models.The outline of this paper is as follows. In section 2 we briefly review the PS model and give the basic operational elements which have to be constructed in an implementation of a classical or quantum PS agent. Then, in section 3 we give a more formal treatment of the standard, classical PS agent, and show explicitly how such an agent may be implemented in an ion trap set-up. In particular, in section 3.3, we discuss how the technique of adding coherent control provides a generic construction for emulating the standard PS agent in quantum systems, specifically in trapped ions. Finally, in section 4, we extend our analysis to quantum PS agents by specifying all required operations and describing their implementation in ion traps. In the appendix we further present a simple example for a quantum PS agent that can be straightforwardly implemented in an ion trap, for which we provide numerical simulations incorporating an appropriate error model. PSThe central component of a PS agent, illustrated in figure 1, is the ECM, which can be formally represented as a stochastic network of clips. Clips represent the units of episodic memory, which consist of memorized percepts, actions and ensuing rewards. The process of PS is triggered by perceptual input that initiates a random walk over the clip space. This walk constitutes the stochastic replay of previously established memories and precedes the initiation of real action. The agent's capability to learn is represented by two mechanisms, (i) the adaption of the transition probabilities between the clips, and (ii) the addition of new clips under compositional principles.More formally, at any instance of time the ECM of an agent can be represented as a directed weighted graph, where the vertices represent the clips, and the weights of the edges represent the transition probabilities, see figure 2. We refer to this graph as the clip network. The random walk, or equivalently, the Markov chain, associated to the process of PS is carried out over the clip network. Finally, the learning aspect of the agent is realized by updating the clip network based on the (re...

show abstract

“…This concept is inspired by occupancy grid maps [20], [21], as opposed to post-exploration maps [22] or topological maps [23]. For our cattle recovery task -and despite their simplicity -grid maps still represent a highly effective tool [19] for exploring the solution space of AI solutions [24], [25].…”

Section: Introduction and Related Workmentioning

confidence: 99%

Aerial Animal Biometrics: Individual Friesian Cattle Recovery and Visual Identification via an Autonomous UAV with Onboard Deep Inference

Andrew

Greatwood

Burghardt

2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

This paper describes a computationally-enhanced M100 UAV platform with an onboard deep learning inference system for integrated computer vision and navigation. The system is able to autonomously find and visually identify by coat pattern individual Holstein Friesian cattle in freely moving herds. We propose an approach that utilises three deep convolutional neural network architectures running live onboard the aircraft: (1) a YOLOv2-based species detector, (2) a dual-stream deep network delivering exploratory agency, and (3) an InceptionV3-based biometric long-term recurrent convolutional network for individual animal identification. We evaluate the performance of each of the components offline, and also online via real-world field tests comprising 147 minutes of autonomous low altitude flight in a farm environment over a dispersed herd of 17 heifer dairy cows. We report errorfree identification performance on this online experiment. The presented proof-of-concept system is the first of its kind. It represents a practical step towards autonomous biometric identification of individual animals from the air in open pasture environments for tag-less AI support in farming and ecology.

show abstract

Projective simulation applied to the grid-world and the mountain-car problem

Cited by 17 publications

References 12 publications

Active learning machine learns to create new quantum experiments

Active learning machine learns to create new quantum experiments

Quantum-enhanced deliberation of learning agents using trapped ions

Aerial Animal Biometrics: Individual Friesian Cattle Recovery and Visual Identification via an Autonomous UAV with Onboard Deep Inference

Contact Info

Product

Resources

About