Control and Guidance of Low-Cost Robots via Gesture Perception for Monitoring Activities in the Home

Sempere, Angel D.; Serna-Leon, Arturo; Gil, Pablo; Méndez, Santiago Timoteo Puente; Torres, Fernando

doi:10.3390/s151229853

Cited by 8 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Raspberry Pi microprocessor has received extensive attention since its release in 2012 due to its powerful function, low price, and convenient operation. It has been widely used in laboratories, education, robots, and other fields [14][15][16][17]. In addition, Raspberry Pi's rich remoteoperated features can greatly reduce experimental bias and error, thus easy for automation to obtain huge amount of accurate dataset [9].…”

Section: Introductionmentioning

confidence: 99%

A novel open-source raspberry Pi-based behavioral testing in zebrafish

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

The zebrafish (Danio rerio) is widely used as a promising high-throughput model organism in neurobehavioral research. The mobility of zebrafish can be dissected into multiple behavior endpoints to assess its neurobehavioral performance. However, such facilities on the market are expensive and clumsy to be used in laboratories. Here, we designed a low-cost, automatic zebrafish behavior assay apparatus, barely without unintentional human operational errors. The data acquisition part, composed of Raspberry Pi and HQ Camera, automatically performs video recording and data storage. Then, the data processing process is also on the Raspberry Pi. Water droplets and inner wall reflection of multi-well cell culture plates (used for placing zebrafish) will affect the accuracy of object recognition. And during the rapid movement of zebrafish, the probability of zebrafish tracking loss increased significantly. Thus, ROI region and related thresholds were set, and the Kalman filter algorithm was performed to estimate the best position of zebrafish in each frame. In addition, all functions of this device are realized by the custom-written behavior analysis algorithm, which makes the optimization of the setup more efficient. Furthermore, this setup was also used to analyze the behavioral changes of zebrafish under different concentrations of alcohol exposure to verify the reliability and accuracy. The alcohol exposure induced an inverted U-shape dose-dependent behavior change in zebrafish, which was consistent with previous studies, showcasing that the data obtained from the setup proposed in this study are accurate and reliable. Finally, the setup was comprehensively assessed by evaluating the accuracy of zebrafish detection (precision, recall, F-score), and predicting alcohol concentration by XGBoost. In conclusion, this study provides a simple, and low-cost package for the determination of multiple behavioral parameters of zebrafish with high accuracy, which could be easily adapted for various other fields.

show abstract

Section: Introductionmentioning

confidence: 99%

A novel open-source raspberry Pi-based behavioral testing in zebrafish

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Marin et al (2014) used a Leap Motion and a Kinect to collect the feature sets of the human hand separately, which were then fed into a multi-class SVM classifier to recognize different static gestures. Sempere et al (2015) used a depth camera to capture the 3D data of the hand and performed gesture analysis and recognition, to control a low-cost robot indoors. For one-hand gesture interaction, the scene is simple and easy to implement.…”

Section: Introductionmentioning

confidence: 99%

“…Although the vision-based gesture interaction has the advantages of simple operation and natural interaction, it still needs to be further improved: For some gesture-based methods, such as references (Marin et al , 2014; Sempere et al , 2015; Fahn and Chu, 2011), static gestures are recognized, which only controls the robot to move in a predefined direction or order the robot to execute the preset commands, resulting in inflexible interaction and low interaction efficiency. As an improvement, dynamic gestures are adopted to flexibly control the robot, such as methods (Kobayashi et al , 2014; Pang, 2015; Jin et al , 2016). However, they are mainly for one-hand gesture interaction and could fail due to gesture occlusion when used for two-hand gesture interaction. Further, references (Du and Zhang, 2015; Sherrah and Gong, 2000; Utsumi and Ohya, 1999) solve the problem of gesture occlusion by different ways.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Gesture-based human-robot interface for dual-robot with hybrid sensors

Zhang

Shen

et al. 2019

View full text Add to dashboard Cite

Purpose The purpose of this paper is the research of a novel gesture-based dual-robot collaborative interaction interface, which achieves the gesture recognition when both hands overlap. This paper designs a hybrid-sensor gesture recognition platform to detect the both-hand data for dual-robot control. Design/methodology/approach This paper uses a combination of Leap Motion and PrimeSense in the vertical direction, which detects both-hand data in real time. When there is occlusion between hands, each hand is detected by one of the sensors, and a quaternion-based algorithm is used to realize the conversion of two sensors corresponding to different coordinate systems. When there is no occlusion, the data are fused by a self-adaptive weight fusion algorithm. Then the collision detection algorithm is used to detect the collision between robots to ensure safety. Finally, the data are transmitted to the dual robots. Findings This interface is implemented on a dual-robot system consisting of two 6-DOF robots. The dual-robot cooperative experiment indicates that the proposed interface is feasible and effective, and it takes less time to operate and has higher interaction efficiency. Originality/value A novel gesture-based dual-robot collaborative interface is proposed. It overcomes the problem of gesture occlusion in two-hand interaction with low computational complexity and low equipment cost. The proposed interface can perform a long-term stable tracking of the two-hand gestures even if there is occlusion between the hands. Meanwhile, it reduces the number of hand reset to reduce the operation time. The proposed interface achieves a natural and safe interaction between the human and the dual robot.

show abstract

“…Vision-based gesture recognition is a critical interface for non-intrusive human-robot interaction (HRI) systems, for which many natural and convenient recognition methods have been proposed [1,2,3]. In a traditional HRI system, a user usually needs to memorize a predefined gesture language or a set of instructions before interaction.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Local Spatiotemporal Features from RGB-D Data for One-Shot Learning Gesture Recognition

Lin

Ruan

et al. 2016

Sensors

View full text Add to dashboard Cite

Noise and constant empirical motion constraints affect the extraction of distinctive spatiotemporal features from one or a few samples per gesture class. To tackle these problems, an adaptive local spatiotemporal feature (ALSTF) using fused RGB-D data is proposed. First, motion regions of interest (MRoIs) are adaptively extracted using grayscale and depth velocity variance information to greatly reduce the impact of noise. Then, corners are used as keypoints if their depth, and velocities of grayscale and of depth meet several adaptive local constraints in each MRoI. With further filtering of noise, an accurate and sufficient number of keypoints is obtained within the desired moving body parts (MBPs). Finally, four kinds of multiple descriptors are calculated and combined in extended gradient and motion spaces to represent the appearance and motion features of gestures. The experimental results on the ChaLearn gesture, CAD-60 and MSRDailyActivity3D datasets demonstrate that the proposed feature achieves higher performance compared with published state-of-the-art approaches under the one-shot learning setting and comparable accuracy under the leave-one-out cross validation.

show abstract

Control and Guidance of Low-Cost Robots via Gesture Perception for Monitoring Activities in the Home

Cited by 8 publications

References 26 publications

A novel open-source raspberry Pi-based behavioral testing in zebrafish

A novel open-source raspberry Pi-based behavioral testing in zebrafish

Gesture-based human-robot interface for dual-robot with hybrid sensors

Adaptive Local Spatiotemporal Features from RGB-D Data for One-Shot Learning Gesture Recognition

Contact Info

Product

Resources

About