Rotary Inverted Pendulum (RIP) plays a vital role in control engineering. Rotary Inverted Pendulum is a complex, nonlinear, non-minimum-phase and under-actuated system which has various applications in the field of Robotics. The main contribution of this paper is to design and control RIP by using Linear Quadratic Regulator (LQR) controller for stabilization at vertically upright position - the unstable equilibrium point, and Linear Quadratic Tracker (LQT) controller for tracking the desired trajectory. Besides, stability of the closed-loop system is analyzed for ensuring the reliability of the developed controller. The simulation is carried out in MATLAB/Simulink environment, and the proposed controllers have been tested on Rotary Inverted Pendulum hardware that is designed by authors. The analysis and results conducted on the system demonstrate the performance of the control schemes, including stabilization of unstable equilibrium point, tracking the desired trajectory, and system response showing the robustness and effectiveness of methods.