Kinematics-based approach for robot programming via human arm motion

Jha, Abhishek; Chiddarwar, Shital S.; Alakshendra, Veer; Andulkar, Mayur V.

doi:10.1007/s40430-016-0662-z

Cited by 10 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many researchers use the Kinect sensor for gesture control of robots [19][20][21][22][23][24][25]. An important part of robot programming is preparing an optimal trajectory and gripping points [26][27][28][29][30][31]. An interesting use of the Kinect sensor is its use as a 3D scanning sensor mounted on a FANUC R2000 industrial robot [32].…”

Section: Introductionmentioning

confidence: 99%

Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode

Kaczmarek

Panasiuk

Borys

et al. 2020

Sensors

View full text Add to dashboard Cite

The paper presents the possibility of using the Kinect v2 module to control an industrial robot by means of gestures and voice commands. It describes the elements of creating software for off-line and on-line robot control. The application for the Kinect module was developed in the C# language in the Visual Studio environment, while the industrial robot control program was developed in the RAPID language in the RobotStudio environment. The development of a two-threaded application in the RAPID language allowed separating two independent tasks for the IRB120 robot. The main task of the robot is performed in Thread No. 1 (responsible for movement). Simultaneously, Thread No. 2 ensures continuous communication with the Kinect system and provides information about the gesture and voice commands in real time without any interference in Thread No. 1. The applied solution allows the robot to work in industrial conditions without the negative impact of the communication task on the time of the robot’s work cycles. Thanks to the development of a digital twin of the real robot station, tests of proper application functioning in off-line mode (without using a real robot) were conducted. The obtained results were verified on-line (on the real test station). Tests of the correctness of gesture recognition were carried out, and the robot recognized all programmed gestures. Another test carried out was the recognition and execution of voice commands. A difference in the time of task completion between the actual and virtual station was noticed; the average difference was 0.67 s. The last test carried out was to examine the impact of interference on the recognition of voice commands. With a 10 dB difference between the command and noise, the recognition of voice commands was equal to 91.43%. The developed computer programs have a modular structure, which enables easy adaptation to process requirements.

show abstract

Section: Introductionmentioning

confidence: 99%

Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode

Kaczmarek

Panasiuk

Borys

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Many researchers use the KINECT sensor for gesture control of robots [16][17][18][19][20][21][22]. Important part of robot programming is preparing an optimal trajectory and gripping points [23][24][25][26][27][28]. An interesting use of the KINECT sensor is its use as a 3D scanning sensor mounted on a FANUC R2000 industrial robot [29].…”

Section: Introductionmentioning

confidence: 99%

Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode

Kaczmarek¹,

Panasiuk²,

Borys³

et al. 2020

Preprint

View full text Add to dashboard Cite

The paper presents the possibility of using KINECT v2 module to control an industrial robot by means of gestures and voice commands. It describes elements of creating software for off-line and on-line robot control. The application for KINECT module was developed in C# language in Visual Studio environment, while the industrial robot control program was developed in RAPID language in RobotStudio environment. The development of a two-threaded application in RAPID language allowed to separate two independent tasks for the IRB120 robot. The main task of the robot is performed in thread no. 1 (responsible for movement). Simultaneously working thread no. 2 ensures continuous communication with the KINECT system and provides information about the gesture and voice commands in real time without any interference in thread no. 1. The applied solution allows the robot to work in industrial conditions without negative impact of communication task on the time of robot’s work cycles. Thanks to the development of a digital twin of the real robot station, tests of proper application functioning in off-line mode (without using a real robot) were conducted. Obtained results were verified online (on the real test station). Tests of correctness of gesture recognition were carried out, the robot recognized all programmed gestures. Another test carried out was the recognition and execution of voice commands. A difference in the time of task completion between the actual and virtual station was noticed - the average difference was 0.67 s. The last test carried out was to examine the impact of interference on the recognition of voice commands. With a 10dB difference between the command and noise, the recognition of voice commands was equal to 91.43%. The developed computer programs have a modular structure, which enables easy adaptation to process requirements.

show abstract

“…1. It could also be used as a reference for programming by demonstration using stereo cameras as an alternative to the Kinect system [15], automatic control of underwater robots [18], and addition of localization capabilities to mobile robots used to inspect power lines [12]. Robust OMR in outdoor environments can be incorporated in end-effector design along with other desirable characteristics [10].…”

Section: Introductionmentioning

confidence: 99%

A robust methodology for outdoor optical mark recognition

Andrade

Trabasso

Eguti

et al. 2017

J Braz. Soc. Mech. Sci. Eng.

View full text Add to dashboard Cite

Outdoor optical mark recognition is an extremely useful tool for recognition of large industrial equipment and application of computer vision-based systems for tracking and positioning. However, current algorithms rely on thresholding and corner detection to identify checkerboard-like patterns, which is not appropriate for non-uniform lighting conditions. This paper presents a robust methodology to identify optical markers in outdoor environments. A GPU-based region filling algorithm automatically detects all contiguous color regions without computing seed points. Post-processing steps extract highlevel information from these regions. Analysis of identified contiguous color region allows simultaneous identification of all checkerboard and targets (concentric regions) in the scene. Analysis of variance demonstrates that the proposed methodology is robust to lighting, environment, perspective, and occlusion. Tests indicate that precision and recall for checkerboard and target identification in outdoor conditions are expected to be above 97%. The parallel algorithm implementation using OpenCL yields better results and is two times faster than previous region filling algorithms, taking about 0.6 s to process a full-HD picture using modern hardware Keywords Computer vision Á Optical mark recognition Á OpenCL Á Parallel processing Á Design of experiments Á Heterogeneous computing

show abstract

Kinematics-based approach for robot programming via human arm motion

Abstract: comparing its performance with a priori approach. The experimental results indicate that the proposed method is reliable, quick and efficient to teach robot from task demonstrations.

Cited by 10 publications

References 20 publications

Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode

Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode

Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode

A robust methodology for outdoor optical mark recognition

Contact Info

Product

Resources

About