With the fast advance of mobile chips technologies, a node in a smart camera network can afford sophisticated processing via on-board multicore CPUs and GPUs, e.g., face detection. The performance of a general purpose face detector, however, may degrade seriously under specific situations with unexpected challenges such as facial coverage or bad illumination. This degradation is due to the difference of probability distributions between training data and testing data, known as source data domain and target data domain, respectively. To better adapt a smart camera network to a specific situation, some form of target domain adaptation is needed, which usually requires both the source domain data and as much as possible target domain data at each node, which may strain storage capacity and bandwidth. In this paper, we propose an adaptation method which fuses the source specific hypotheses (SSHs) and target specific hypotheses (TSHs) -requiring only a pre-trained face detector and a few target data to be shared by peer-to-peer communications, thus relieving the storage and bandwidth constraints. The method uses the "accuracy-regularization" objective as the adaptation model, to fuse SSHs and TSHs, and tries to minimize the misclassification error on target data. With an existing frontal face detector, we conduct experiments to verify our algorithm, covering cases of video surveillance, extreme pose challenge, and different illumination spectra. Significant performance gains are observed with only dozens of target data in all the experiments, demonstrating the effectiveness of the proposed adaptation model. Therefore, the proposed adaption can be applied to a smart camera network with peer-to-peer communications to improve the network's overall face detection performance.