We present a novel method to approximate optimal feedback laws for nonlinear optimal control based on low‐rank tensor train (TT) decompositions. The approach is based on the Dirac–Frenkel variational principle with the modification that the optimization uses an empirical risk. Compared to current state‐of‐the‐art TT methods, our approach exhibits a greatly reduced computational burden while achieving comparable results. A rigorous description of the numerical scheme and demonstrations of its performance are provided.