Most of the existing studies on controllability have only focused on networks with non‐negative weighted interactions/edges among agents. In fact, there are both positive and negative weighted interactions in a network, where positive/negative weight implies cooperative/antagonistic interaction. The controllability of leader‐based multi‐agent systems described by directed/undirected signed graphs is respectively considered for fixed topology and switching topology. A quantitative characterization of controllability is given based on the influence pattern of leaders and some algebraic and graphical features of controllability are established according to consensus protocols. Moreover, it is proved that all follower agents can be steered to the desired configuration under structurally balanced conditions. Specially, a simultaneously structurally balanced signed multiagent network can be controllable on switching topology even though each subnetwork is uncontrollable. Finally, several numerical examples and simulation results are given.