Game controllers have been planned and improved throughout the years to be as easy to understand as could reasonably be expected. A game controller is a gadget utilized with games or theatre setups to give contribution to a computer game, commonly to control an item or character in the game. Information gadgets that have been named game controllers incorporate consoles, mice, gamepads, joysticks, and so on. A few controllers are intended to be purposely best for one sort of game, for example, guiding wheels for driving games, move cushions for moving games, and light firearms for firing games. The aim here is to create a virtual environment, where the user is appealed by various gesture controls in a gaming application. A Gesture is an action that has to be seen or felt by someone else (here a PC) and has to convey some piece of information. Now obviously, to create a virtual gaming environment, we need to create a real-time gaming application first. We’ll be designing our 2D and 3D gaming applications through Unity 3D video game engine. The data used in this project is primarily from the Ego Hands dataset. After an input has been taken, and the consequent action has been performed, we’ll use this activity for future development of the model by using Tensor-Flow. The input will be taken through the webcam of the PC which will be accessed and combined to the gaming application and hands dataset by WebGL. WebGL is a JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser without the use of plug-ins.