This is the second in a series of papers that exploit the physical coupling of tethered spacecraft to derive a propellant-free spin-up and attitude control strategy. We take a nonlinear control approach to underactuated tethered formation flying spacecraft, whose lack of full state feedback linearizability, along with their complex nonholonomic behavior, characterizes the difficult nonlinear control problem. We introduce several nonlinear control laws that are more efficient in tracking time-varying trajectories than linear control. We also extend our decentralized control approach to underactuated tethered systems, thereby eliminating the need for any inter-satellite communication. To our knowledge, this work reports the first nonlinear control results for underactuated tethered formation flying spacecraft. This article further illustrates the potential of the proposed strategy by providing a new momentum dumping method that does not use torque-generating thrusters.