Humans teaming with intelligent autonomous agents are becoming indispensable in work environments.However, human-agent teams pose significant challenges as team dynamics are complex arising from the task and social aspects of human-agent interactions. To improve our understanding of human-agent team dynamics, we conducted a systematic literature review. Drawing on Mathieu et al.'s (2019) teamwork model developed for all-human teams, we map the landscape of research to human-agent team dynamics, including structural features, compositional features, mediating mechanisms, and the interplay of the above features and mechanisms. We reveal that the development of human-agent team dynamics is still nascent, with a particular focus on information sharing, trust development, agents' human likeness behaviors, shared cognitions, situation awareness, and function allocation. Gaps remain in many areas of team dynamics, such as team processes, adaptability, shared leadership, and team diversity. We offer various interdisciplinary pathways to advance research on human-agent teams.