Identification of novel protein binding sites expands «druggable genome» and opens new opportunities for drug discovery. Generally, presence or absence of a binding site depends on the three-dimensional conformation of a protein, making binding site identification resemble to object detection problem in computer vision. Here we introduce a computational approach for the largescale detection of protein binding sites, named BiteNet, that considers protein conformations as the 3D-images, binding sites as the objects on these images to detect, and conformational ensembles of proteins as the 3D-videos to analyze. BiteNet is suitable for spatiotemporal detection of hard-tospot allosteric binding sites, as we showed for conformation-specific binding site of the epidermal growth factor receptor, oligomer-specific binding site of the ion channel, and binding sites in G protein-coupled receptors. BiteNet outperforms state-of-the-art methods both in terms of accuracy and speed, taking about 1.5 minute to analyze 1000 conformations of a protein with ∼ 2000 atoms. BiteNet is available at https://github.com/i-Molecule/bitenet.