Human and Robotic Fish Interaction Controlled Using Hand Gesture Image Processing

Angani, Amarnathvarma; Lee, Jin Wook; Talluri, Teressa; Lee, Jae-Young; Shin, Kyoo Jae

doi:10.18494/sam.2020.2925

Cited by 7 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After detecting the color of the robot fish, the position of the corresponding robot fish will be identified and, by using this position data, we can track the robot fish using color marks. (16)(17)(18)(19)(20)(21) The robot fish stop swimming when they encounter a color mark and continue swimming when the color mark is removed. The coordinates of 3D fish were recorded by sensors, and these coordinates were used to identify the position of fish.…”

Section: Methodsmentioning

confidence: 99%

Realization of 3D Aqua Hologram Augmented Reality of Robot Fish

Angani¹,

Talluri²,

Lee³

et al. 2020

Sensors and Materials

Self Cite

View full text Add to dashboard Cite

This paper is about the realization of robot fish motion along with 3D hologram fish in an aquarium, with the augmented reality of 3D aqua holograms to attract viewers. In this new proposed concept, the animated fish video with 3D hologram is projected into a robot fish aquarium. However, this process of projecting 3D hologram fish into a robot fish aquarium may cause the overlap of the robot fish with the animated fish. To avoid this problem of overlapping, sensors play an important role. Image sensors track the robot fish movement. The proposed concept can be realized by boundary detection and optical flow algorithms, which are used to detect the positions of the robot fish. The performance of this detection algorithm was evaluated and satisfactory results were obtained. This 3D hologram with augmented reality has been successfully installed at

show abstract

Section: Methodsmentioning

confidence: 99%

Realization of 3D Aqua Hologram Augmented Reality of Robot Fish

Angani¹,

Talluri²,

Lee³

et al. 2020

Sensors and Materials

Self Cite

View full text Add to dashboard Cite

show abstract

“…Altering fish swarm patterns to manipulate the fish behavior in order to increase human interaction is rather a newer area of research. Biomimicry in robotic fish has been vastly improved while lesser attention has been given to the use of a robotic fish to change the behavior of fish to improve human interaction [ 35 , 36 ]. In addition to the use of robotic fish in the exploration and examination of aquatic life, another benefit of robotic fish is to improve human interaction with fish.…”

Section: Literature Reviewmentioning

confidence: 99%

Altering Fish Behavior by Sensing Swarm Patterns of Fish in an Artificial Aquatic Environment Using an Interactive Robotic Fish

Manawadu

Zoysa

Perera

et al. 2023

Sensors

View full text Add to dashboard Cite

Numerous studies have been conducted to prove the calming and stress-reducing effects on humans of visiting aquatic environments. As a result, many institutions have utilized fish to provide entertainment and treat patients. The most common issue in this approach is controlling the movement of fish to facilitate human interaction. This study proposed an interactive robot, a robotic fish, to alter fish swarm behaviors by performing an effective, unobstructed, yet necessary, defined set of actions to enhance human interaction. The approach incorporated a minimalistic but futuristic physical design of the robotic fish with cameras and infrared (IR) sensors, and developed a fish-detecting and swarm pattern-recognizing algorithm. The fish-detecting algorithm was implemented using background subtraction and moving average algorithms with an accuracy of 78%, while the swarm pattern detection implemented with a Convolutional Neural Network (CNN) resulted in a 77.32% accuracy rate. By effectively controlling the behavior and swimming patterns of fish through the smooth movements of the robotic fish, we evaluated the success through repeated trials. Feedback from a randomly selected unbiased group of subjects revealed that the robotic fish improved human interaction with fish by using the proposed set of maneuvers and behavior.

show abstract

“…(4), u and v are respectively the horizontal and vertical directions of the optical flow, which are also known as motion vectors. (14)(15)(16)(17) To compute the motion vectors, the optical flow limitation equation [Eq. (5)] is used with the Lucas-Kanade method.…”

Section: Optical Flow Algorithmmentioning

confidence: 99%

Lead-lag Swimming Control of Robot Fish Using Color Detection Algorithm

Angani¹,

Kang²,

Talluri³

et al. 2020

Sensors and Materials

Self Cite

View full text Add to dashboard Cite

This paper is about lead-lag robot fish swimming control by image processing. In robot fish aquariums, fish can generally move randomly in any path. This irregular movement may cause collisions between the fish, damaging them. Hence, an effective swimming control method is necessary. We thus proposed a lead-lag swimming control system for robot fish. Here, we simply study the detection of moving objects in an aquarium because we need to find the positions of moving robot fish. The locations of the robot fish are recognized using an image processing technique employing image and position sensors. This method is used to obtain the velocity for each pixel in an image and assumes a constant velocity in each video frame to obtain the positions of robot fish by comparing consecutive video frames. By using the position data, we compute the distance between robot fish and determine which robot fish is the lead fish and which fish is the lagging fish. The lead fish then waits for the lagging fish to catch up. The results of this proposed system are satisfactory in preventing collisions between robot fish. This system is exhibited in Busan Science Museum in South Korea.

show abstract

Human and Robotic Fish Interaction Controlled Using Hand Gesture Image Processing

Cited by 7 publications

References 10 publications

Realization of 3D Aqua Hologram Augmented Reality of Robot Fish

Realization of 3D Aqua Hologram Augmented Reality of Robot Fish

Altering Fish Behavior by Sensing Swarm Patterns of Fish in an Artificial Aquatic Environment Using an Interactive Robotic Fish

Lead-lag Swimming Control of Robot Fish Using Color Detection Algorithm

Contact Info

Product

Resources

About