This paper presents an automatic soccer annotation system. The system consists of 3 modules: 3D reconstruction module, behavior analysis module and annotation module. The first module reconstructs the 3D model of the soccer scene, including positions, velocities and acceleration of each player, and the trajectory of ball. According to these parameters and domain knowledge, behavior analysis module will recognize each player's action with a finite state machine. Then, annotation module will convert key player's behavior into annotation words. The system proves to be robust when errors of 3D reconstruction results are small.