In this paper, an adaptive approach is presented to nonlinear system for real-time robotic visual tracking of a moving polyhedral object. A light stripe vision system consists of a laser-stripe sensor and a CCD the camera fixed to robot end-effector, and projects planar light on the polyhedral object.The geometric conditions can be provided to assure location of features of the polyhedral faces. The objective is to predict the location of features of the object on the image plane based on the light stripe vision system and then to determine an optimal control input that will move the camera so that the image features align with their desired positions. We first give the equations of observation and state-space by using the motion rules of the camera and the object. Then, the system can be represented as an MIMO ARMAX model and an efficient estimation model. The estimation model can process on-line estimation of the 3D related parameters between the camera and the object. Those parameters are used as on-line train values of neural network. The control scheme adopts a neural-PID controller that can adjust the PID controller parameters. The paper concludes with the simulation results and the computer simulation shows that the proposed method is effective to visual tracking of combining vision and control.