Small cantilevers with a megahertz-order resonance frequency provide excellent sensitivity and speed in liquid-environment atomic force microscopy (AFM). However, stable and accurate oscillation control of a small cantilever requires the photothermal excitation, which has hindered their applications to the studies on photo-sensitive materials. Here, we develop a magnetic excitation system with a bandwidth wider than 4 MHz, enabling a light-free excitation of small cantilevers. In the system, a cantilever with a magnetic bead is driven by a magnetic field generated by a coil. In the coil driver, a differentiation circuit is used for compensating the frequency dependence of the coil impedance and keeping the current constant. By implementing several differentiation circuits with different frequency ranges, we enable to drive various cantilevers having different resonance frequencies with sufficient excitation efficiency. In contrast to the conventional coil driver with a closed-loop circuit, the developed one consists of an open-loop circuit and hence can be stably operated regardless of the coil design. With the developed system, atomic-resolution imaging of mica in liquid using a small cantilever with a megahertz-order resonance frequency is demonstrated. This development should lead to the future applications of AFM with small cantilevers to the studies on various photo-sensitive materials and phenomena.