Tactile sensing is a key sensor modality for robots interacting with their surroundings. These sensors provide a rich and diverse set of data signals that contain detailed information collected from contacts between the robot and its environment. The data is however not limited to individual contacts and can be used to extract a wide range of information about the objects in the robots environment as well as the robots own actions during the interactions.In this paper, we provide an overview of tactile information and its applications in robotics. We present a hierarchy consisting of raw, contact, object, and action levels to structure the tactile information, with higher-level information often building upon lower-level information. We discuss different types of information that can be extracted at each level of the hierarchy. The paper also includes an overview of different types of robot applications and the types of tactile information that they employ.The paper concludes with a discussion of tactile-based computational framework and future tactile applications which are still beyond current robot's capabilities.

Abstract-During grasping and other in-hand manipulation tasks maintaining a stable grip on the object is crucial for the task's outcome. Inherently connected to grip stability is the concept of slip. Slip occurs when the contact between the fingertip and the object is partially lost, resulting in sudden undesired changes to the objects state. While several approaches for slip detection have been proposed in the literature, they frequently rely on previous knowledge of the manipulated object. This previous knowledge may be unavailable, seeing that robots operating in real-world scenarios often must interact with previously unseen objects.In our work we explore the generalization capabilities of well known supervised learning methods, using random forest classifiers to create generalizable slip predictors. We utilize these classifiers in the feedback loop of an object stabilization controller. We show that the controller can successfully stabilize previously unknown objects by predicting and counteracting slip events.

Accurate object shape knowledge provides important information for performing stable grasping and dexterous manipulation. When modeling an object using tactile sensors, touching the object surface at a fixed grid of points can be sample inefficient. In this paper, we present an active touch strategy to efficiently reduce the surface geometry uncertainty by leveraging a probabilistic representation of object surface. In particular, we model the object surface using a Gaussian process and use the associated uncertainty information to efficiently determine the next point to explore. We validate the resulting method for tactile object surface modeling using a real robot to reconstruct multiple, complex object surfaces.

Controlling contact with arbitrary, unknown objects defines a fundamental problem for robotic grasping and in-hand manipulation. In real-world scenarios, where robots interact with a variety of objects, the sheer number of possible contact interactions prohibits acquisition of the necessary models for all objects of interest. As an alternative to traditional control approaches that require accurate models, predicting the onset of slip can enable controlling contact interactions without explicit model knowledge. In this article, we propose a grip stabilization approach for novel objects based on slip prediction. Using tactile information, such as applied pressure and fingertip deformation, our approach predicts the onset of slip and modulates the contact forces accordingly. We formulate a supervised-learning problem to predict the future onset of slip from high-dimensional tactile information provided by a BioTac sensor. This slip mapping generalizes across objects, including objects absent during training. We evaluate how different input features, slip prediction time horizons, and available tactile information channels, impact prediction accuracy. By mounting the sensor on a PA-10 robotic arm, we show that employing prediction in a controller's feedback loop yields an object grip stabilization controller that can successfully stabilize multiple, previously unknown objects by counteracting slip events.

In this work we introduce a non-planar soft high-resolution tactile sensor. An iteration of the GelSight sensors, it enables future GelSights to have more complicated form factors, such as a humanoid fingertip. To do this we introduce a novel method for achieving directional lighting along the entirety of a curved sensor using light piping. Light piping uses total internal reflection and a semi-specular membrane to constrain the path of the light inside the sensor until the sensing membrane is deformed. By using this new membrane and changing the geometry, we introduce a new bidirectional reflectance distribution function and new optics. This require new calibration procedures in the form of developing a fisheye projection model, and developing a neighborhood and location based continuous look-up table to map the relationship between RGB value and surface normal orientation of the membrane at a point. Finally we perform two dexterous manipulation task with feedback from the sensors in the form of controlled rolling of an object on a support surface, and lid removal off a jar. We also give instructions on how to manufacture the sensor as well as increasing the durability of the membrane for all GelSight sensors.