Key characteristics of thermoelectric, ballistic thermionic and quasi diffusive thermionic energy converters are compared. First, the main assumptions used to derive the linear Boltzmann transport equations for electrons are examined and the possibility that a higher order transport coefficient may become relevant is discussed. In the linear transport regime, there is a fundamental trade off between high Seebeck coefficient and high electrical conductivity for bulk materials and for many multilayer structures due to the interplay between electronic density-of-states (DOS) and electron group velocity and also due to the shape of DOS versus energy curve deep inside a band. While low dimensional structures alter the density-of-states, a similar trade off still exists. If large barrier heights and high doping concentrations could be achieved solid-state thermionic energy converters would be able to alleviate this trade off, thereby achieving a very high thermoelectric power factor. For this to occur, the electron transverse momentum perpendicular to heterostructure barriers must not be conserved. This can be achieved with non-planar structures or with embedded nanostructures. Finally, a comparison between thermoelectric/ thermionic devices and thermophotovoltaic energy converters shows a difference in the average energy of the emitted hot carriers due to the difference between electronic and photonic density-of-states in the reservoirs. The use of both electrons and photons from a hot reservoir or the engineering of the reservoir density-ofstates may provide additional means to achieve higher efficiency in energy conversion devices and to approach the limit given by the entropy generation more easily.