Real world data, especially in the domain of robotics, is notoriously costly to collect. One way to circumvent this can be to leverage the power of simulation to produce large amounts of labelled data. However, training models on simulated images does not readily transfer to realworld ones. Using domain adaptation methods to cross this "reality gap" requires a large amount of unlabelled realworld data, whilst domain randomization alone can waste modeling power. In this paper, we present Randomizedto-Canonical Adaptation Networks (RCANs), a novel approach to crossing the visual reality gap that uses no realworld data. Our method learns to translate randomized rendered images into their equivalent non-randomized, canonical versions. This in turn allows for real images to also be translated into canonical sim images. We demonstrate the effectiveness of this sim-to-real approach by training a vision-based closed-loop grasping reinforcement learning agent in simulation, and then transferring it to the real world to attain 70% zero-shot grasp success on unseen objects, a result that almost doubles the success of learning the same task directly on domain randomization alone. Additionally, by joint finetuning in the real-world with only 5,000 real-world grasps, our method achieves 91%, attaining comparable performance to a state-of-the-art system trained with 580,000 real-world grasps, resulting in a reduction of real-world data by more than 99%.

Spectral measurements in the infrared (IR) optical range provide unique fingerprints of materials which are useful for material analysis, environmental sensing, and health diagnostics 1 . Current IR spectroscopy techniques require the use of optical equipment suited for operation in the IR range, which faces challenges of inferior performance and high cost. Here we develop a spectroscopy technique, which allows spectral measurements in the IR range using visible spectral range components. The technique is based on nonlinear interference of infrared and visible photons, produced via Spontaneous Parametric Down Conversion (SPDC) 2,3 . The intensity interference pattern for a visible photon depends on the phase of an IR photon, which travels through the media. This allows determining properties of the media in the IR range from the measurements of visible photons. The technique can substitute and/or complement conventional IR spectroscopy techniques, as it uses well-developed optical components for the visible range.Entangled photons continue to play a crucial role in advancing many areas of quantum technologies, including cryptography 4,5 , computing 6,7 and metrology 8,9 . They can be obtained using a variety of methods, with SPDC in non-linear optical crystals, being well established 10 . We consider a specific type of interference technique, referred to as a non-linear interferometry, which is analogue to a conventional Mach-Zehnder or Young interferometer but with the two splitting mirrors being substituted with two SPDC crystals 3 . In a non-linear interferometer, two SPDC crystals are pumped by a common laser, so that down-converted photons (signal and idler) from one crystal are injected into the second crystal. Signal and idler photons from the two crystals interfere and produce a distinctive interference pattern in frequency and spatial domains. Depending on experimental configuration one can observe interference either in intensity or in the second-order correlation function [11][12][13] .

Large language models can encode a wealth of semantic knowledge about the world. Such knowledge could be extremely useful to robots aiming to act upon high-level, temporally extended instructions expressed in natural language. However, a significant weakness of language models is that they lack real-world experience, which makes it difficult to leverage them for decision making within a given embodiment. For example, asking a language model to describe how to clean a spill might result in a reasonable narrative, but it may not be applicable to a particular agent, such as a robot, that needs to perform this task in a particular environment. We propose to provide real-world grounding by means of pretrained skills, which are used to constrain the model to propose natural language actions that are both feasible and contextually appropriate. The robot can act as the language model's "hands and eyes," while the language model supplies high-level semantic knowledge about the task. We show how low-level skills can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally extended instructions, while value functions associated with these skills provide the grounding necessary to connect this knowledge to a particular physical environment. We evaluate our method on a number of real-world robotic tasks, where we show the need for real-world grounding and that this approach is capable of completing long-horizon, abstract, natural language instructions on a mobile manipulator. The project's website and the video can be found at say-can.github.io. (a) Large Language Models (LLMs) (b) SayCan

Resonant metasurfaces are an attractive platform for enhancing the non-linear optical processes, such as second harmonic generation (SHG), since they can generate very large local electromagnetic fields while relaxing the phase-matching requirements. Here, we take this platform a step closer to the practical applications by demonstrating visible range, continuous wave (CW) SHG. We do so by combining the attractive material properties of gallium phosphide with engineered, high quality-factor photonic modes enabled by bound states in the continuum. For the optimum case, we obtain efficiencies around 5e-5 % W −1 when the system is pumped at 1200 nm wavelength