The concept of concurrent material and structure optimization aims at alleviating the computational discovery of optimum microstructure configurations in multiphase hierarchical systems, whose macroscale behavior is governed by their microstructure composition that can evolve over multiple length scales from a few micrometers to centimeters. It is based on the split of the multiscale optimization problem into two nested sub-problems, one at the macroscale (structure) and the other at the microscales (material). In this paper, we establish a novel formulation of concurrent material and structure optimization for multiphase hierarchical systems with elastoplastic constituents at the material scales. Exploiting the thermomechanical foundations of elastoplasticity, we reformulate the material optimization problem based on the maximum plastic dissipation principle such that it assumes the format of an elastoplastic constitutive law and can be efficiently solved via modified return mapping algorithms. We integrate continuum micromechanics based estimates of the stiffness and the yield criterion into the formulation, which opens the door to a computationally feasible treatment of the material optimization problem. To demonstrate the accuracy and robustness of our framework, we define new benchmark tests with several material scales that, for the first time, become computationally feasible. We argue that our formulation naturally extends to multiscale optimization under further path-dependent effects such as viscoplasticity or multiscale fracture and damage.