Considering speaking ability is one of the important abilities that students have to master in learning English. However, most of their speaking ability is low. It is caused by several factors, one of those factors is affective factors (motivation, anxiety, and self-confidence). The research aims to know what affective factors that mostly influence students’ speaking ability. This research applied quantitative design with simple linear regression analysis. The sample of the research is 39 students from two classes X science class and X social class at SMA Muhammadiyah 9 Surabaya. Questionnaire and speaking test used as the instrument of the research. The result of the speaking test showed that the students’ speaking ability is below average. It is proven by the scores of students speaking tests are mostly below the minimal completeness criteria that is said by the school. The result shows that among three affective factors, the most influential factor is self-confidence, and the second influential factor is motivation. Meanwhile anxiety is proven not to influence the students speaking ability. The findings show that the total significant value of motivation is 0,011, anxiety is 0.063, and self-confidence is 0.007. It shows that speaking ability is influenced by motivation and self-confidence, but the result of the regression test shows that anxiety does not influence students' speaking ability even though the data result of anxiety is appropriate with the theory of the research. It can be caused by various aspects such as students’ characteristics, class atmosphere during the speaking test, and teacher attitude. So, the teacher should be fun when setting up the atmosphere of speaking, no pressure, and make students less anxious, so they don’t feel to be tested and to be assessed.