Robotic fabric manipulation has applications in cloth and cable management, senior care, surgery and more. Existing fabric manipulation techniques, however, are designed for specific tasks, making it difficult to generalize across different but related tasks. We address this problem by extending the recently proposed Visual Foresight framework to learn fabric dynamics, which can be efficiently reused to accomplish a variety of different fabric manipulation tasks with a single goalconditioned policy. We introduce VisuoSpatial Foresight (VSF), which extends prior work by learning visual dynamics on domain randomized RGB images and depth maps simultaneously and completely in simulation. We experimentally evaluate VSF on multi-step fabric smoothing and folding tasks both in simulation and on the da Vinci Research Kit (dVRK) surgical robot without any demonstrations at train or test time. Furthermore, we find that leveraging depth significantly improves performance for cloth manipulation tasks, and results suggest that leveraging RGBD data for video prediction and planning yields an 80% improvement in fabric folding success rate over pure RGB data. Supplementary material is available at https://sites.google.com/ view/fabric-vsf/.

No abstract

When operating in unstructured environments such as warehouses, homes, and retail centers, robots are frequently required to interactively search for and retrieve specific objects from cluttered bins, shelves, or tables. Mechanical Search describes the class of tasks where the goal is to locate and extract a known target object. In this paper, we formalize Mechanical Search and study a version where distractor objects are heaped over the target object in a bin. The robot uses an RGBD perception system and control policies to iteratively select, parameterize, and perform one of 3 actions -push, suction, grasp -until the target object is extracted, or either a time limit is exceeded, or no high confidence push or grasp is available. We present a study of 5 algorithmic policies for mechanical search, with 15,000 simulated trials and 300 physical trials for heaps ranging from 10 to 20 objects. Results suggest that success can be achieved in this long-horizon task with algorithmic policies in over 95% of instances and that the number of actions required scales approximately linearly with the size of the heap. Code and supplementary material can be found at http://ai.stanford.edu/mech-search. * Authors have contributed equally and names are in alphabetical order.

Intraocular pressure (IOP) is a key clinical parameter in glaucoma management. However, despite the potential utility of daily measurements of IOP in the context of disease management, the necessary tools are currently lacking, and IOP is typically measured only a few times a year. Here we report on a microscale implantable sensor that could provide convenient, accurate, ondemand IOP monitoring in the home environment. When excited by broadband near-infrared (NIR) light from a tungsten bulb, the sensor's optical cavity reflects a pressure-dependent resonance signature that can be converted to IOP. NIR light is minimally absorbed by tissue and is not perceived visually. The sensor's nanodot-enhanced cavity allows for a 3-5 cm readout distance with an average accuracy of 0.29 mm Hg over the range of 0-40 mm Hg. Sensors were mounted onto intraocular lenses or silicone haptics and secured inside the anterior chamber in New Zealand white rabbits. Implanted sensors provided continuous in vivo tracking of short-term transient IOP elevations and provided continuous measurements of IOP for up to 4.5 months.

In earthquake early warning (EEW), every sufficiently impulsive signal is potentially the first evidence for an unfolding large earthquake. More often than not, however, impulsive signals are mere nuisance signals. One of the most fundamental-and difficult-tasks in EEW is to rapidly and reliably discriminate real local earthquake signals from all other signals. This discrimination is necessarily based on very little information, typically a few seconds worth of seismic waveforms from a small number of stations. As a result, current EEW systems struggle to avoid discrimination errors and suffer from false and missed alerts. In this study we show how modern machine learning classifiers can strongly improve real-time signal/noise discrimination. We develop and compare a series of nonlinear classifiers with variable architecture depths, including fully connected, convolutional and recurrent neural networks, and a model that combines a generative adversarial network with a random forest. We train all classifiers on the same data set, which includes 374 k local earthquake records (M3.0-9.1) and 946 k impulsive noise signals. We find that all classifiers outperform existing simple linear classifiers and that complex models trained directly on the raw signals yield the greatest degree of improvement. Using 3-s-long waveform snippets, the convolutional neural network and the generative adversarial network with a random forest classifiers both reach 99.5% precision and 99.3% recall on an independent validation data set. Most misclassifications stem from impulsive teleseismic records, and from incorrectly labeled records in the data set. Our results suggest that machine learning classifiers can strongly improve the reliability and speed of EEW alerts.Plain Language Summary Seismic stations record not only earthquake signals but also a wide variety of nuisance signals. Some of these nuisance signals are impulsive and can initially look very similar to real earthquake signals. This is a problem for earthquake early warning (EEW) algorithms, which sometimes misinterpret such signals as being real earthquake signals, and which may then send out false alerts. For each registered impulsive signal, EEW systems need to decide (or, classify) in real-time whether or not the signal stems from an actual earthquake. State-of-the-art machine learning (ML) classifiers have been shown to strongly outperform more standard linear classifiers in a wide range of classification problems. Here we analyze the performance of a variety of different ML classifiers to identify which type of classifier leads to the most reliable signal/noise discrimination in an EEW context. We find that we can successfully train complex deep learning classifiers that can discriminate between nuisance and earthquake signals very reliably (accuracy of 99.5%). Less complex ML classifiers also outperform a linear classifier, but with significantly higher error rates. The deep ML classifiers may allow EEW systems to almost entirely avoid false and missed signal detections...