This study aims to optimally control the level of a four-tank system at the steady state in the random disturbance environment using the stochastic approximation (SA) approach. Firstly, the stochastic optimal control problem of an equivalent discrete-time is introduced, where the voltages to the pumps are the control inputs. By minimizing the sum of squared errors, the liquid levels are estimated. Then, first-order necessary conditions are derived by defining the Hamiltonian function. Thus, the optimal voltages are calculated based on the estimated liquid levels to update the gradient of the cost function. Finally, for illustration, parameters in the system are considered and a simulation is conducted. The simulation results show that the state estimation and control law design can perform well, and the liquid levels are addressed along the steady state. In conclusion, the applicability of the SA approach for handling a four-tank system with random disturbances is demonstrated.

<abstract> <p>In this paper, optimal control of an inverted pendulum on a cart system is studied. Since the nonlinear structure of the system is complex, and in the presence of random disturbances, optimization and control of the motion of the system become more challenging. For handling this system, a discrete-time stochastic optimal control problem for the system is described, where the external force is considered as the control input. By defining a loss function, namely, the mean squared errors to be minimized, the stochastic approximation (SA) approach is applied to estimate the state dynamics. In addition, the Hamiltonian function is defined, and the first-order necessary conditions are derived. The gradient of the cost function is determined so that the SA approach is employed to update the control sequences. For illustration, considering the values of the related parameters in the system, the discrete-time stochastic optimal control problem is solved iteratively by using the SA algorithm. The simulation results show that the state estimation and the optimal control law design are well performed with the SA algorithm, and the motion of the inverted pendulum cart is addressed satisfactorily. In conclusion, the efficiency of the SA approach for solving the inverted pendulum on a cart system is verified.</p> </abstract>