This paper presents a novel adaptive Fast Motion Estimation (FME) Algorithm, which reduces the number of search points, computational complexity and therefore lowering power consumption, while providing improved quality per watt in FME. The algorithm minimizes computation and storage and from there, power consumption, while trying to preserve quality as much as possible. It is very suitable for hardware implementation, as in a motion co-processor. The pervasiveness of mobile devices with HD video capabilities demands such low power hardware accelerators. The proposed algorithm is a novel method, that can identify the best search pattern within a given region of the HD video frame, based on its motion dynamics to achieve lower power video encoding. The goal is to achieve the best quality with the minimum number of search iterations. The reduced number of checks translates into power savings. The motivation of the presented algorithm is to learn the motion dynamics within a region from a small subset of blocks within that region and apply that knowledge to the remaining large set of blocks in the same region. This adaptiveness makes it robust with respect to the nature of the video. Thus it works equally well with more dynamic and with less dynamic video sequences. The results show that the proposed algorithm can reduce computations by about 4 times compared to other fixed search patterns algorithms, while staying within 1 dB of PSNR results. That equates to about 75% power savings at the expense of not more than 1 dB of PSNR quality loss.