In this work we develop a numerical method for solving a type of convex graph-structured tensor optimization problems. This type of problems, which can be seen as a generalization of multi-marginal optimal transport problems with graph-structured costs, appear in many applications. In particular, we show that it can be used to model and solve nonlinear density control problems, including convex dynamic network flow problems and multi-species potential mean field games. The method is based on coordinate ascent in a Lagrangian dual, and under mild assumptions we prove that the algorithm converges globally. Moreover, under a set of stricter assumptions, the algorithm converges R-linearly. To perform the coordinate ascent steps one has to compute projections of the tensor, and doing so by brute force is in general not computationally feasible. Nevertheless, for certain graph structures we derive efficient methods for computing these projections. In particular, these graph structures are the ones that occur in convex dynamic network flow problems and multi-species potential mean field games. We also illustrate the methodology on numerical examples from these problem classes.