Software development productivity has been widely studied in academia from different perspectives, including its meaning, measurement, dimensions, and how to improve it, at an individual, team, and organizational level. In the last decade, several factors have emerged that influence the way of working and thus, the productivity of agile development teams. These factors could be related to interactive, collaborative, and simplified work environments, among others. A better understanding of such factors and how they affect productivity could help organizations to determine how to deal with them when implementing agile practices and where to focus management efforts to have better results and deliver value faster. However, an insufficient empirical basis on this topic was found. This work shows an analysis of the state of the art regarding the factors that influence these teams, studying their impact on productivity, all this through a systematic literature review process. After analyzing results, it can be concluded that the factors that most affect team productivity are those related to the interaction between members, especially communication and distance, as well as the way in which they are organized. Those aspects related to the individual characteristics and soft skills are influential factors over the team performance. It is important to point out that even though the studies mention factors that affect productivity in different ways, they do not propose strategies for adapting agile practices, and only a couple of them propose practices that have a positive impact.