This paper studies controllability of networked systems in which subsystems are of general high-order linear dynamics and coupled through relative state variables, from a structure perspective. The purpose is to search conditions for subsystem dynamics and subsystem interaction topologies, under which there exists a set of weights for the interaction links such that the associated networked system can be controllable (i.e., structural controllability). Three types of subsystem interaction fashions are considered, which are 1) each subsystem is singleinput-single-output (SISO), 2) each subsystem is multiple-inputmultiple-output (MIMO), and the interaction weights for different channels between two subsystems can be different, and 3) each subsystem is MIMO but the interaction weights between two subsystems are the same. Necessary and/or sufficient conditions for structural controllability are given. These conditions indicate that, under certain conditions on the subsystem dynamics, the whole system is structurally controllable, if and only if the network topology is globally input-reachable. Finally, these results are extended to the case where subsystem dynamics are fixed but the interaction topologies are switching. A promising point of the structure analysis taken in this paper is that, it can handle certain subsystem heterogeneities, which are illustrated by some practical systems, including the liquid-level systems, the power networks and the mechanical systems.