Real-Time Object Detection and Grasping for a Robotic Arm

Abstract

Research and development with robotic arms have been explored for their potential to perform various physical tasks such as sorting items by colors and shapes. Prior work and research made advancements with robotic arm systems by combining them with deep neural networks (DNN) and computer-based vision to detect and interact with objects. These systems take a massive amount of time to deploy and develop due to time-consuming DNN training costs. This training cost staggers the development of robotic arms since computer vision needs a DNN to operate.Many of these systems rely on object detection to pick up items from a fixed camera overhead within their field of vision. Fixed overhead cameras compensate for the limitation of computer vision to properly locate and estimate the distances of the object in 3D space. Such that an overhead camera will assume detected objects will be on the surfaces directly in front. Research to estimate distance with computer vision relies on well-trained DNN to estimate accurately. Computer vision distance estimation has had yet to be successful with robotic arm systems. This project aims to reduce the training cost on DNN by using Transfer-Learning, to promptly deploy a well-trained DNN to a robotic arm system to detect an object and estimate its distance from a non-fixed field of vision to grasp objects in real-time. v Preface and/or Acknowledgements I want to take the time to appreciate and show my gratitude to my advisor: Dr. Qin. He is a fantastic professor and educator who inspires his students and challenges and motivates them to strive to achieve their career goals. When starting this research project, I was asked, "What is it that you want to do for a career? What is your dream?". I answered that I was interested in robotics and Deep Learning & Computer Vision which I had little knowledge about. Together we came up and developed this research project considering my interest and what he can offer his knowledge on. There were many hurdles and obstacles, especially with learning a handful of new subjects, programming language, software, developing an entire system with practically zero experience.Dr.Qin has been a beacon of knowledge and support throughout the entire phase of this research, and I am genuinely grateful to have been able to develop and learn from this challenging project as well. I also wanted to give the time to appreciate Dr.Quintero as a valuable committee member for this research. He has been an excellent resource and guidance on working with a robotic system for this project's scope. His extensive knowledge and professional experience in robotics make it an honor to showcase my work and receive his opinions and thoughts on the system. vi Table of Contents LIST OF TABLES .