The inverse Brayton cycle is a potential technology for waste heat energy recovery. It consists of three components: one turbine, one heat exchanger, and one compressor. The exhaust gas is further expanded to subatmospheric pressure in the turbine, and then cooled in the heat exchanger, last compressed in the compressor into the atmosphere. The process above is the reverse of the pressurized Brayton cycle. This work has presented the strategy on performance improvement of the inverse Brayton cycle system for energy recovery in turbocharged diesel engines, which has pointed the way to the future development of the inverse Brayton cycle system. In the paper, an experiment was presented to validate the numerical model of a 2.0 l turbocharged diesel engine. Meanwhile, the influence laws of the inverse Brayton cycle system critical parameters, including turbocharger speed and efficiencies, and heat exchanger efficiency, on the system performance improvement for energy recovery are explored at various engine operations. The results have shown that the engine exhaust energy recovery efficiency increases with the engine speed up, and it has a maximum increment of 6.1% at the engine speed of 4000 r/min (the engine rated power point) and the full load. At the moment, the absolute pressure was before final compression is 51.9 kPa. For the inverse Brayton cycle system development in the future, it is essential to choose a more effective heat exchanger. Moreover, variable geometry turbines are very appropriate to achieve a proper matching between the turbocharging system and the inverse Brayton cycle system.