Accurate pedestrian trajectory prediction is crucial in many fields. This requires the full use and learning of pedestrians’ social interactions, movements, and environmental information. In view of the current research on pedestrian trajectory prediction, wherein most of the pedestrian interaction information is explored from the level of overall interaction, this paper proposes the SISGAN model, which designs a social interaction module from the perspective of the target pedestrian, and takes four kinds of interaction information as the influencing factors of pedestrian interaction, so as to describe the influence mechanism of pedestrian–pedestrian interaction. In addition, in terms of environmental information, the index density of pedestrian historical trajectory in space is taken into account in the extraction of environmental information, which increases the potential correlation between environmental information and pedestrians. Finally, we integrate social interaction information and environmental information and make the final trajectory prediction based on GAN. Experiments on ETH and UCY datasets demonstrate the effectiveness of the SISGAN model proposed in this paper.