An optimal trajectory planner for humanoid robots with a haptically controlled arm is proposed. This trajectory planner simultaneously plans time-varying arm wrench constraints, foot step positions, and a Center-of-Mass (COM) trajectory. This system has the useful characteristic of ensuring that the robot does not fall when the human operator commands an excessively large arm wrench and in the long term the resultant arm wrench approaches the desired arm wrench via a combination of changes in the foot positions and the COM position. This is accomplished via a Model Predictive Controller with a Quadratic Programming-based trajectory optimizer that calculates a time series of arm wrench constraints, foot positions and, a COM trajectory that optimizes the criteria of minimizing the tracking error of the desired foot positions and the desired arm wrench. A simulation and experiments demonstrate the validity of the proposed algorithm.