Retrosplenial cortex has been implicated in processing sensory information and spatial learning. Abnormal neural activity in the retrosplenial cortex is found to be associated with psychedelics, as well as in mouse and non-human primate models of autism spectrum disorders (ASD). However, whether the retrosplenial cortex may directly regulate social behaviors remained unclear. In this work, we found that the neural activity of retrosplenial agranular cortex (RSA), a subregion of retrosplenial cortex, was initially activated, then quickly suppressed upon social contact. Interestingly, the up-down phase of RSA neurons is essential for normal social behaviors. We showed that PV-positive GABAergic neurons in the hippocampal CA1 region sent inhibitory projections to RSA. Blockade of the CA1-RSA inhibitory inputs significantly compromised social behavior. Remarkably, enhancing the CA1-RSA inhibitory input was able to rescue defects of social behaviors in a mouse model for ASD. This work not only suggests a neural mechanism for salience processing of social behavior, but also provides a candidate brain region for ASD intervention with neural modulation approaches.