In recent years, wireless vision sensor network (WVSN) is being used to retrieve video content from different image sensors which are connected to different devices wirelessly. This information is used to do video analysis which can help to automate different tasks such as video surveillance. For such systems, power consumption during processing and communicating information has been a challenge because of limited energy sources at the node. To deal with the energy consumption problem, in this paper WVSN is proposed with its algorithm and hardware implementation for a smart home application. The computation tasks have been divided between the sensor node and the central server. While taking care of privacy issues during the transmission of data, a low complexity system has been developed for sensor nodes. For video analysis, foreground segmentation, object labeling, and object tracking have been performed. An efficient binary data compression technique has been proposed to compress the information during the labeling process. The proposed system has a high recognition rate for gesture recognition and human tracking. The system can achieve eight frames per second during processing information on Raspberry Pi board. This system can be extended further to include other smart home applications.