High current fast electrons at the megaampere level provide a unique way to generate high energy density states of matter, which are related to many applications. However, the large divergence angle of the fast electrons typically over 50 degrees is a significant disadvantage for applications. The guiding effect by the self-generated azimuthal magnetic fields of the fast electron current is found to be very limited due to their cone-shaped spatial structure. In this work, we present a new understanding on the collimation conditions of fast electrons under such a magnetic field structure. It is shown that the transverse peak position of the magnetic field layer plays a crucial role to collimate the fast electrons rather than its magnitude. Based upon this, a new two-pulse collimating scheme is proposed, where a guiding precursor pulse is adopted to form proper azimuthal magnetic fields and a main pulse is for fast electron generation as usual. The present scheme can be implemented relatively easily with the precursor lasers at the 10 TW level with a duration of two hundred femtoseconds, with which the divergence angle of fast electrons driven by the main pulse can be confined within a few degrees. Our scheme can find practical applications in high energy density science.