Disability may limit someone to move freely, especially when the severity of the disability is high. In order to help disabled people control their wheelchair, head movement-based control is preferred due to its reliability. This paper proposed a head direction detector framework which can be applied to wheelchair control. First, face and nose were detected from a video frame using Haar cascade classfier. Then, the detected bounding boxes were used to initialize Kernelized Correlation Filters tracker. Direction of a head was determined by relative position of the nose to the face, extracted from tracker's bounding boxes. Results show that the method effectively detect head direction indicated by 82% accuracy and very low detection or tracking failure.