The main contribution of this paper is to analytically solve the Hamilton-Jacobi-Bellman equation for a class of third order nonlinear optimal control problems for which the dynamics are affine and the cost is quadratic in the input. The proposed solution method is based on the notion of inverse optimality with a variable part of the cost to be determined in the solution. One special advantage of the proposed method is that the solution is directly obtained for the control input without the computation of a cost function first. The cost can however also be obtained based on the control input. Furthermore, a Lyapunov function can be constructed for a subclass of optimal control problems, yielding a proof certificate for stability. Experimental results of a path following problem of a unicycle are also presented.