Computer Vision is a more and more pervasive technology in nowadays image and video processing applications: examples include image driven search, stereoscopical matching, panorama stitching and industrial automation. Compact Descriptors for Visual Search (CDVS) is an algorithm for Computer Vision recently proposed as part of the MPEG-7 standard: it has the ability to select points of interest in the image (also referred to as keypoints) that exhibit robustness, in a certain degree, with respect to changes like homogeneous variations in luminance, changes in point of view, rotations, rescaling and geometrical distortion of the image. Keypoint Refinement is a phase of the CDVS algorithm which is aimed at discarding candidate keypoints that are likely to be unstable for their algebraic properties. This paper presents an FPGA circuit design that implements this phase on fixed point data with real time compatible throughput. Implementation results show a negligible impact on resources allocation even on mid-sized FPGAs