This paper is devoted to robust integral sliding‐mode consensus tracking control for leader‐following multi‐agent systems with higher order, time‐varying delay and disturbances under fixed and switching topologies. A delay‐dependent sufficient condition for the existence of integral sliding surfaces is given in terms of linear matrix inequalities. This guarantees the global stability of the dynamics in the specified switching surface and the consensus tracking of multi‐agent systems is achieved. Unlike the previous works, both the controller gain and sliding mode matrix are designed but not chosen in this note, which increases the design degree of freedom, and moreover, the reciprocally convex combination technique is given to get a less conservative stability condition for multi‐agent systems with arbitrary time‐derivative of the time‐varying delay. Illustrative examples with fixed and switching topologies are provided to test the effectiveness of the proposed method.