Tracked vehicles have been widely used in construction, agriculture, and the military. Major problems facing the industry, however, are high emissions and fuel consumption. Hybrid electric tracked vehicles have thus become increasingly popular because of their improved fuel economy and reduced emissions. While the series hybrid system has drawn the most attention and has been applied in most cases, the low efficiency caused by energy conversion losses and large propulsion motors has limited its development. A novel multi-mode powertrain with two output shafts controlling each side of the track independently is first proposed. The powertrain is a three-planetary-gear power-split system with one engine, three motors, and an ultracapacitor pack. Compared with the existing technologies, the proposed powertrain can realize skid steering without an extra steering mechanism, and significantly improve the overall efficiency. To demonstrate the advantages of the novel powertrain, a topology-control-size integrated optimization problem is solved based on drivability, fuel economy, and cost. Final simulation results show that the optimized design with downsized components can produce about a 30% improvement in drivability and a 15% improvement in fuel economy compared with the commonly used series hybrid benchmark. Moreover, the optimized design is verified to be much more economical taking cumulative cost into account, which is very attractive for potential industrial applications in the future.