In this paper, the authors propose a markerless visual servo controller for a micro helicopter with an onboard wireless camera. The state of the helicopter is derived by a robust visual tracking algorithm based on phase correlation. The tracking algorithm is useful over a tiled carpet and a uniform carpet where other visual tracking algorithms do not work. The proposed controller achieves long-time hovering flights over both of the tiled carpet and the uniform carpet. The controller does not require any specified markers, prior texture information, or any settings other than flat floors.