We have proposed "Attentive Workbench (AWB)," a deskwork support system. Using cameras, projectors, and selfmoving trays, the system supports human user from both physical and informational aspects. In this paper, an interface between the user and AWB based on pointing gestures is developed. Considering the usage of AWB, a simple and reliable interface with quick response is implemented. Deskwork support using the present interface and real self-moving trays are demonstrated.