This paper develops an online adaptive critic algorithm based on policy iteration for partially unknown nonlinear optimal control with infinite horizon cost function. In the proposed method, only a critic network is established, which eliminates the action network, to simplify its architecture. The online least squares support vector machine (LS-SVM) is utilized to approximate the gradient of the associated cost function in the critic network by updating the input-output data. Additionally, a data buffer memory is added to alleviate computational load. Finally, the feasibility of the online learning algorithm is demonstrated in simulation on two example systems.