Perceptually-guided deep neural networks for ego-action prediction: Object grasping

González-Díaz, Iván; Benois-Pineau, Jenny; Domenger, Jean‐Philippe; Cattaert, Daniel; Rugy, Aymar de

doi:10.1016/j.patcog.2018.11.013

Cited by 42 publications

(40 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In this way, several prototypes were produced and proofs of concepts were developed in order to illustrate potential use cases in various fields in relation with human-driven robotics. As a broadly connectable platform, it allows to investigate hybrid control strategies, combining biomechanical signals with motion- or eye-tracking tools and computer vision techniques (de San Roman et al, 2017; González-Díaz et al, 2019). Reachy can also help study how vision-based control strategies would help driving rehabilitation devices, such as an assistive arm fixed to a wheelchair, for use by patients suffering from Spinal Cord Injury (SCI) (Corbett et al, 2013, 2014).…”

Section: Discussionmentioning

confidence: 99%

“…When maintaining gaze on the target object, the geometry in a dynamic scene is also unstable due to micro-saccades. This is why a filtering of gaze fixation signal along the time is needed (González-Díaz et al, 2019). Moreover, today a localization of objects in a gaze-predicted area can be solved together with an object-recognition task, employing powerful deep CNN classifiers.…”

Section: Proofs Of Conceptmentioning

confidence: 99%

See 1 more Smart Citation

Reachy, a 3D-Printed Human-Like Robotic Arm as a Testbed for Human-Robot Control Strategies

Mick

Lapeyre²,

Rouanet³

et al. 2019

Front. Neurorobot.

Self Cite

View full text Add to dashboard Cite

To this day, despite the increasing motor capability of robotic devices, elaborating efficient control strategies is still a key challenge in the field of humanoid robotic arms. In particular, providing a human “pilot” with efficient ways to drive such a robotic arm requires thorough testing prior to integration into a finished system. Additionally, when it is needed to preserve anatomical consistency between pilot and robot, such testing requires to employ devices showing human-like features. To fulfill this need for a biomimetic test platform, we present Reachy, a human-like life-scale robotic arm with seven joints from shoulder to wrist. Although Reachy does not include a poly-articulated hand and is therefore more suitable for studying reaching than manipulation, a robotic hand prototype from available third-party projects could be integrated to it. Its 3D-printed structure and off-the-shelf actuators make it inexpensive relatively to the price of an industrial-grade robot. Using an open-source architecture, its design makes it broadly connectable and customizable, so it can be integrated into many applications. To illustrate how Reachy can connect to external devices, this paper presents several proofs of concept where it is operated with various control strategies, such as tele-operation or gaze-driven control. In this way, Reachy can help researchers to explore, develop and test innovative control strategies and interfaces on a human-like robot.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Proofs Of Conceptmentioning

confidence: 99%

Reachy, a 3D-Printed Human-Like Robotic Arm as a Testbed for Human-Robot Control Strategies

Mick

Lapeyre²,

Rouanet³

et al. 2019

Front. Neurorobot.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, we extend our experimental work by studying alternative dimensionality reduction techniques in order to examine the validity of our assumptions. Furthermore, we applied camera ego-motion compensation, as in [20], to examine the improvement it may bestow upon our best models for both descriptors. We accumulated and present activity recognition results for each class, in the form of confusion matrices, and examine how each class performs depending on object detection performance.…”

Section: Experimental Workmentioning

confidence: 99%

First-person activity recognition from micro-action representations using convolutional neural networks and object flow histograms

Giannakeris

Petrantonakis

Avgerinakis

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

A novel first-person human activity recognition framework is proposed in this work. Our proposed methodology is inspired by the central role moving objects have in egocentric activity videos. Using a Deep Convolutional Neural Network we detect objects and develop discriminant object flow histograms in order to represent fine-grained micro-actions during short temporal windows. Our framework is based on the assumption that large scale activities are synthesized by fine-grained micro-actions. We gather all the micro-actions and perform Gaussian Mixture Model clusterization, so as to build a microaction vocabulary that is later used in a Fisher encoding schema. Results show that our method can reach 60% recognition rate on the benchmark ADL

show abstract

“…This is also the case of lifeLog data [23]. While for specific tasks in lifelog data new Deep Architectures are being designed [24], the standard backbones such as ResNet [25] are suitable for recognition of egocentric scenes [26]. Nevertheless, data from risky situations are not very frequent in typical lifeLog data or in commonly used datasets [27].…”

Section: Deep Learning In Lifelog Visual Content Miningmentioning

confidence: 99%

Detection of Semantic Risk Situations in Lifelog Data for Improving Life of Frail People

Yebda

Benois‐Pineau

Pech

et al. 2020

Proceedings of the 2020 International Conference on Multimedia Retrieval

Self Cite

View full text Add to dashboard Cite

The automatic recognition of risk situations for frail people is an urgent research topic for the interdisciplinary artificial intelligence and multimedia community. Risky situations can be recognized from lifelog data recorded with wearable devices. In this paper, we present a new approach for the detection of semantic risk situations for frail people in lifelog data. Concept matching between general lifelog and risk taxonomies was realized and tuned AlexNet was deployed for detection of two semantic risks situations such as risk of domestic accident and risk of fraud with promising results. CCS CONCEPTS • Applied computing → Health care information systems; • Computing methodologies → Neural networks.

show abstract

Perceptually-guided deep neural networks for ego-action prediction: Object grasping

Cited by 42 publications

References 59 publications

Reachy, a 3D-Printed Human-Like Robotic Arm as a Testbed for Human-Robot Control Strategies

Reachy, a 3D-Printed Human-Like Robotic Arm as a Testbed for Human-Robot Control Strategies

First-person activity recognition from micro-action representations using convolutional neural networks and object flow histograms

Detection of Semantic Risk Situations in Lifelog Data for Improving Life of Frail People

Contact Info

Product

Resources

About